[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 506669976 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002294] x2apic enabled [ 0.003005] Switched APIC routing to physical x2apic. [ 0.004009] kvm-guest: setup PV IPIs [ 0.007448] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008016] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009010] pid_max: default: 32768 minimum: 301 [ 0.010127] LSM: Security Framework initializing [ 0.011047] Yama: becoming mindful. [ 0.012025] SELinux: Initializing. [ 0.013059] *** VALIDATE selinux *** [ 0.021593] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025579] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026157] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027100] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028103] *** VALIDATE tmpfs *** [ 0.030286] *** VALIDATE proc *** [ 0.031199] *** VALIDATE cgroup *** [ 0.032006] *** VALIDATE cgroup2 *** [ 0.034089] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035134] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036005] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037021] Spectre V2 : User space: Vulnerable [ 0.038006] Speculative Store Bypass: Vulnerable [ 0.041030] debug: unmapping init [mem 0xffffffff88e59000-0xffffffff88e60fff] [ 0.043275] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044680] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045021] ... version: 2 [ 0.046009] ... bit width: 48 [ 0.047010] ... generic registers: 4 [ 0.048009] ... value mask: 0000ffffffffffff [ 0.049011] ... max period: 00007fffffffffff [ 0.050011] ... fixed-purpose events: 3 [ 0.051009] ... event mask: 000000070000000f [ 0.053217] rcu: Hierarchical SRCU implementation. [ 0.055572] smp: Bringing up secondary CPUs ... [ 0.056591] x86: Booting SMP configuration: [ 0.057017] .... node #0, CPUs: #1 #2 #3 [ 0.067059] smp: Brought up 1 node, 4 CPUs [ 0.069009] smpboot: Max logical packages: 1 [ 0.070013] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.097494] node 0 deferred pages initialised in 26ms [ 0.102148] devtmpfs: initialized [ 0.103265] x86/mm: Memory block size: 128MB [ 0.106198] gcov: version magic: 0x41383552 [ 0.108192] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.109170] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.110253] pinctrl core: initialized pinctrl subsystem [ 0.111176] [ 0.111853] ************************************************************* [ 0.112013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.113011] ** ** [ 0.114014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.115014] ** ** [ 0.116014] ** This means that this kernel is built to expose internal ** [ 0.117015] ** IOMMU data structures, which may compromise security on ** [ 0.118013] ** your system. ** [ 0.119011] ** ** [ 0.120012] ** If you see this message and you are not debugging the ** [ 0.121011] ** kernel, report this immediately to your vendor! ** [ 0.122011] ** ** [ 0.123012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.124014] ************************************************************* [ 0.125793] NET: Registered protocol family 16 [ 0.126574] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.127046] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.128050] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.129651] cpuidle: using governor menu [ 0.131203] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.134866] PCI: Using configuration type 1 for base access [ 0.137186] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.144119] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.145000] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.147006] cryptd: max_cpu_qlen set to 1000 [ 0.149239] ACPI: Added _OSI(Module Device) [ 0.150000] ACPI: Added _OSI(Processor Device) [ 0.150000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.150000] ACPI: Added _OSI(Processor Aggregator Device) [ 0.157865] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.167021] ACPI: Interpreter enabled [ 0.168033] ACPI: PM: (supports S0 S3 S4 S5) [ 0.170010] ACPI: Using IOAPIC for interrupt routing [ 0.172100] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.175689] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.190252] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.192030] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.194015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.197066] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.202470] acpiphp: Slot [2] registered [ 0.204121] acpiphp: Slot [5] registered [ 0.205099] acpiphp: Slot [6] registered [ 0.207132] acpiphp: Slot [7] registered [ 0.208364] acpiphp: Slot [8] registered [ 0.210117] acpiphp: Slot [9] registered [ 0.212107] acpiphp: Slot [10] registered [ 0.214121] acpiphp: Slot [3] registered [ 0.216069] acpiphp: Slot [4] registered [ 0.217173] acpiphp: Slot [11] registered [ 0.218057] acpiphp: Slot [12] registered [ 0.220063] acpiphp: Slot [13] registered [ 0.221059] acpiphp: Slot [14] registered [ 0.222183] acpiphp: Slot [15] registered [ 0.223000] acpiphp: Slot [16] registered [ 0.223000] acpiphp: Slot [17] registered [ 0.225093] acpiphp: Slot [18] registered [ 0.226171] acpiphp: Slot [19] registered [ 0.228061] acpiphp: Slot [20] registered [ 0.229067] acpiphp: Slot [21] registered [ 0.230062] acpiphp: Slot [22] registered [ 0.232063] acpiphp: Slot [23] registered [ 0.233091] acpiphp: Slot [24] registered [ 0.234090] acpiphp: Slot [25] registered [ 0.236071] acpiphp: Slot [26] registered [ 0.238079] acpiphp: Slot [27] registered [ 0.239095] acpiphp: Slot [28] registered [ 0.241085] acpiphp: Slot [29] registered [ 0.243156] acpiphp: Slot [30] registered [ 0.245095] acpiphp: Slot [31] registered [ 0.247084] PCI host bridge to bus 0000:00 [ 0.248014] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.251176] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.254021] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.258019] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.261026] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.264025] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.267258] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.274151] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.280000] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.296783] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.302937] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.307018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.310030] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.315024] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.318516] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.327234] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.331057] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.335057] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.344017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.357018] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.360911] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.365385] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.373014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.386017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.410016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.429000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.437013] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.442000] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.443000] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.443000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.443000] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.454024] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.479140] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.484000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.546014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.549013] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.559015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.565867] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.571014] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.575015] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.589014] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.597752] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.605000] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.609012] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.621013] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.630544] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.631000] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.632267] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.634372] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.636228] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.639382] iommu: Default domain type: Passthrough [ 0.642499] SCSI subsystem initialized [ 0.645148] ACPI: bus type USB registered [ 0.647183] usbcore: registered new interface driver usbfs [ 0.649072] usbcore: registered new interface driver hub [ 0.651057] usbcore: registered new device driver usb [ 0.653135] pps_core: LinuxPPS API ver. 1 registered [ 0.656011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.659052] PTP clock support registered [ 0.661381] EDAC MC: Ver: 3.0.0 [ 0.662807] PCI: Using ACPI for IRQ routing [ 0.663925] NetLabel: Initializing [ 0.665009] NetLabel: domain hash size = 128 [ 0.667007] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.670072] NetLabel: unlabeled traffic allowed by default [ 0.672521] vgaarb: loaded [ 0.674758] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.677011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.687227] clocksource: Switched to clocksource kvm-clock [ 0.840072] VFS: Disk quotas dquot_6.6.0 [ 0.841168] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.844109] *** VALIDATE ramfs *** [ 0.845624] *** VALIDATE hugetlbfs *** [ 0.847229] pnp: PnP ACPI init [ 0.849971] pnp: PnP ACPI: found 6 devices [ 0.868628] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.874654] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.877832] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.880166] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.882479] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.884758] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.888182] NET: Registered protocol family 2 [ 0.891157] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.896876] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.900729] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.907099] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.911606] TCP: Hash tables configured (established 65536 bind 65536) [ 0.915051] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.920228] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.923529] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.927814] NET: Registered protocol family 1 [ 0.930339] RPC: Registered named UNIX socket transport module. [ 0.933473] RPC: Registered udp transport module. [ 0.936083] RPC: Registered tcp transport module. [ 0.938309] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.943916] NET: Registered protocol family 44 [ 0.946074] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.950750] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.954701] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.957582] PCI: CLS 0 bytes, default 64 [ 0.961528] Unpacking initramfs... [ 3.023261] debug: unmapping init [mem 0xffff9045bcc54000-0xffff9045bffbffff] [ 3.027637] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.029808] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.032441] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.983746] Initialise system trusted keyrings [ 3.987137] Key type blacklist registered [ 3.989485] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 4.026571] zbud: loaded [ 4.038681] *** VALIDATE nfs *** [ 4.042581] *** VALIDATE nfs4 *** [ 4.047802] pstore: using deflate compression [ 4.056589] Platform Keyring initialized [ 4.181714] NET: Registered protocol family 38 [ 4.182927] Key type asymmetric registered [ 4.184180] Asymmetric key parser 'x509' registered [ 4.186146] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.188837] io scheduler mq-deadline registered [ 4.190239] io scheduler kyber registered [ 4.191721] io scheduler bfq registered [ 4.193643] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.196106] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.198396] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.200928] ACPI: Power Button [PWRF] [ 4.206060] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.215456] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.233291] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 4.241585] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.258619] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.287307] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.315461] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.320433] Non-volatile memory driver v1.3 [ 4.321892] Linux agpgart interface v0.103 [ 4.356252] virtio_blk virtio1: [vda] 134592 512-byte logical blocks (68.9 MB/65.7 MiB) [ 4.359204] vda: detected capacity change from 0 to 68911104 [ 4.379526] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.382614] vdb: detected capacity change from 0 to 1073741824 [ 4.409546] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.412259] vdc: detected capacity change from 0 to 2621440000 [ 4.425386] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.427937] vdd: detected capacity change from 0 to 2621440000 [ 4.443546] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.446558] vde: detected capacity change from 0 to 4294967296 [ 4.461476] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.464574] vdf: detected capacity change from 0 to 4294967296 [ 4.471455] libphy: Fixed MDIO Bus: probed [ 4.476925] usbcore: registered new interface driver usbserial_generic [ 4.479638] usbserial: USB Serial support registered for generic [ 4.482057] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.486931] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.488937] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.491490] mousedev: PS/2 mouse device common for all mice [ 4.494230] rtc_cmos 00:05: RTC can wake from S4 [ 4.498818] rtc_cmos 00:05: registered as rtc0 [ 4.502848] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.503443] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.510524] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.517439] intel_pstate: CPU model not supported [ 4.517888] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.527704] hid: raw HID events driver (C) Jiri Kosina [ 4.529672] usbcore: registered new interface driver usbhid [ 4.531392] usbhid: USB HID core driver [ 4.532890] drop_monitor: Initializing network drop monitor service [ 4.535063] Initializing XFRM netlink socket [ 4.536826] NET: Registered protocol family 10 [ 4.539658] Segment Routing with IPv6 [ 4.541131] NET: Registered protocol family 17 [ 4.543160] mpls_gso: MPLS GSO support [ 4.548655] RAS: Correctable Errors collector initialized. [ 4.550857] AVX version of gcm_enc/dec engaged. [ 4.552665] AES CTR mode by8 optimization enabled [ 4.672094] sched_clock: Marking stable (4672068888, 0)->(5782311825, -1110242937) [ 4.677911] registered taskstats version 1 [ 4.681426] Loading compiled-in X.509 certificates [ 4.684276] zswap: loaded using pool lzo/zbud [ 4.728575] Key type big_key registered [ 4.749503] Key type encrypted registered [ 4.752462] ima: No TPM chip found, activating TPM-bypass! [ 4.756086] ima: Allocated hash algorithm: sha1 [ 4.759251] ima: No architecture policies found [ 4.761273] evm: Initialising EVM extended attributes: [ 4.763831] evm: security.selinux [ 4.768465] evm: security.ima [ 4.769566] evm: security.capability [ 4.771243] evm: HMAC attrs: 0x1 [ 4.778891] rtc_cmos 00:05: setting system clock to 2026-01-10 01:21:37 UTC (1768008097) [ 4.797232] debug: unmapping init [mem 0xffffffff89e03000-0xffffffff89ffffff] [ 4.806916] debug: unmapping init [mem 0xffffffff88b82000-0xffffffff88e58fff] [ 4.818062] Write protecting the kernel read-only data: 28672k [ 4.823757] debug: unmapping init [mem 0xffffffff87203000-0xffffffff873fffff] [ 4.828702] debug: unmapping init [mem 0xffffffff87b14000-0xffffffff87bfffff] [ 4.922838] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.954830] systemd[1]: Detected virtualization kvm. [ 4.963903] systemd[1]: Detected architecture x86-64. [ 4.969985] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 5.030959] systemd[1]: No hostname configured. [ 5.040566] systemd[1]: Set hostname to . [ 5.048954] random: systemd: uninitialized urandom read (16 bytes read) [ 5.059886] systemd[1]: Initializing machine ID from random generator. [ 5.907066] random: ln: uninitialized urandom read (6 bytes read) [ 6.548405] random: systemd: uninitialized urandom read (16 bytes read) [ 6.551750] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 6.560613] random: systemd: uninitialized urandom read (16 bytes read) [ 6.563960] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 6.569544] random: systemd: uninitialized urandom read (16 bytes read) [ 6.573721] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local File Systems. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Swap. Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Reached target Paths. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 7.735879] device-mapper: uevent: version 1.0.3 [ 7.737766] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 11.652136] virtio_net virtio0 ens2: renamed from eth0 [ 11.711654] random: fast init done [ 12.489129] scsi host0: ata_piix [ 12.627522] scsi host1: ata_piix [ 12.630074] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 12.635966] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 17.398404] random: crng init done [ 17.405120] random: 5 urandom warning(s) missed due to ratelimiting [ 20.727920] dracut-initqueue[590]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 22.047636] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 24.230520] printk: systemd: 26 output lines suppressed due to ratelimiting [ 25.023679] SELinux: Disabled at runtime. [ 25.195551] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 25.208589] systemd[1]: Detected virtualization kvm. [ 25.213491] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 26.385796] systemd[1]: initrd-switch-root.service: Succeeded. [ 26.392468] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 26.407176] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 26.412394] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 26.419801] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 26.427349] systemd[1]: Starting Journal Service... Starting Journal Service... [ 26.435412] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Switch Root. Mounting Kernel Debug File System... [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on Process Core Dump Socket. Mounting POSIX Message Queue File System... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Remount Root and Kernel File Systems... Mounting Huge Pages File System... Activating swap /dev/disk/by-label/SWAP... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target RPC Port Mapper. [[ 26.889361] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS  OK ] Listening on udev Control Socket. Starting Apply Kernel Variables... Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 27.654033] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 28.638602] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 28.826774] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 29.361624] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 29.404503] EDAC sbridge: Ver: 1.1.2 [ 33.489458] Key type dns_resolver registered [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (7s / no limit)[ 34.412860] NFS: Registering the id_resolver key type [ 34.414800] Key type id_resolver registered [ 34.424833] Key type id_legacy registered [*** ] A start job is running for Configur…-only root support (8s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Login Service... [ OK ] Started irqbalance daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg443-server login: [ 75.660136] libcfs: loading out-of-tree module taints kernel. [ 75.692231] Key type ._llcrypt registered [ 75.694444] Key type .llcrypt registered [ 75.762416] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 85.167296] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 85.825028] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 85.833978] alg: No test for adler32 (adler32-zlib) [ 86.910271] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 87.408962] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 89.079299] Key type lgssc registered [ 89.896209] Lustre: Echo OBD driver; http://www.lustre.org/ [ 97.959147] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 98.712617] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 103.526857] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 107.351435] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 111.129121] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 119.105160] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 126.102896] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 126.135401] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 126.146571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 127.275812] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 127.297048] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 127.336458] Lustre: lustre-MDT0000: new disk, initializing [ 127.397945] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 127.422395] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 129.385076] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 136.403657] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 136.448188] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 136.483537] Lustre: 6485:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 136.500139] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 136.502918] Lustre: Skipped 1 previous similar message [ 136.546310] Lustre: lustre-MDT0001: new disk, initializing [ 136.597477] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 136.614502] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 136.624183] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 138.438167] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 141.055974] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 146.400581] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 146.458822] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 146.585327] Lustre: lustre-OST0000: new disk, initializing [ 146.588385] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 146.616967] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 149.343551] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 152.571101] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 152.578219] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 152.635560] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 157.654396] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 157.688269] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 157.731159] Lustre: lustre-OST0001: new disk, initializing [ 157.735061] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 157.762018] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 158.026698] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 158.039651] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 158.068320] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 160.024961] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 167.570858] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 168.114263] hrtimer: interrupt took 2658009 ns [ 171.510877] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 179.851859] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing check_logdir /tmp/testlogs/ [ 181.985670] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing yml_node [ 184.268153] Lustre: DEBUG MARKER: Client: 2.17.0.RC3 [ 185.342585] Lustre: DEBUG MARKER: MDS: 2.17.0.RC3 [ 186.545390] Lustre: DEBUG MARKER: OSS: 2.17.0.RC3 [ 187.350131] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Jan 9 20:24:38 EST 2026 [ 197.024939] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 197.898229] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 213.471758] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 213.481775] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 213.496755] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 214.501825] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 214.504736] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 214.519158] Lustre: Skipped 1 previous similar message [ 214.523740] Lustre: Skipped 1 previous similar message [ 217.268278] LustreError: 12185:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 217.358825] Lustre: server umount lustre-MDT0000 complete [ 219.655642] LustreError: 6478:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768008312 with bad export cookie 7576124747667579076 [ 219.656058] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 219.660642] LustreError: 6478:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 224.736561] LustreError: 6494:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 224.736679] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 224.737613] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 224.737623] Lustre: Skipped 2 previous similar messages [ 224.749337] LustreError: 6494:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 224.753525] Lustre: Skipped 2 previous similar messages [ 225.811199] LustreError: 12387:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 225.812972] LustreError: 12387:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 225.889568] Lustre: server umount lustre-MDT0001 complete [ 227.967071] LustreError: 12588:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 227.969771] LustreError: 12588:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 228.003992] Lustre: server umount lustre-OST0000 complete [ 230.274537] LustreError: 12788:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 230.280349] LustreError: 12788:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 230.333750] Lustre: server umount lustre-OST0001 complete [ 234.295672] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 240.604557] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 247.146947] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 252.041690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 256.205845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 260.396449] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 264.954424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 264.996948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 265.168438] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 265.195597] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 265.253590] Lustre: lustre-MDT0000: new disk, initializing [ 265.311946] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 265.326635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 268.145746] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 275.445162] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 275.493988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 275.590765] Lustre: 14970:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 275.630972] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 275.634077] Lustre: Skipped 1 previous similar message [ 275.689680] Lustre: lustre-MDT0001: new disk, initializing [ 275.766021] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 275.817681] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 275.823064] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 278.635440] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 284.262987] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 287.001725] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 291.483606] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 291.543146] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 291.693109] Lustre: lustre-OST0000: new disk, initializing [ 291.697180] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 292.903726] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 292.908591] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 292.977857] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 295.698902] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 301.008688] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 303.604348] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 303.733953] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 306.538637] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 306.681053] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 308.197730] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 308.198786] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 308.208187] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 308.223997] Lustre: Skipped 2 previous similar messages [ 313.316897] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 313.323111] Lustre: Skipped 1 previous similar message [ 314.228183] LustreError: 17906:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 314.231160] LustreError: 17906:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 314.313534] Lustre: server umount lustre-OST0000 complete [ 316.900576] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 316.905897] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 316.914422] Lustre: Skipped 1 previous similar message [ 322.764129] LustreError: 18107:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 322.767393] LustreError: 18107:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 322.804332] Lustre: server umount lustre-MDT0000 complete [ 323.552647] LustreError: 14981:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 323.563811] LustreError: 14981:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 325.017945] LustreError: 16715:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768008417 with bad export cookie 7576124747667580609 [ 325.018879] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 325.024345] LustreError: 16715:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 325.222632] Lustre: server umount lustre-MDT0001 complete [ 328.381797] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 20:26:59 (1768008419) [ 332.681581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 332.985031] LustreError: 18836:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 333.079566] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 333.082435] Lustre: Skipped 1 previous similar message [ 335.706282] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 338.401591] LustreError: 18835:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 338.411606] LustreError: 18835:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 340.659971] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 343.036552] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 345.959650] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 348.084184] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 352.513340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 356.391559] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 359.763769] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 371.595742] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 376.293548] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 376.295252] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 376.303963] Lustre: Skipped 3 previous similar messages [ 376.311023] Lustre: Skipped 5 previous similar messages [ 377.810889] LustreError: 21160:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 377.813869] LustreError: 21160:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 377.850767] Lustre: server umount lustre-MDT0000 complete [ 380.033376] LustreError: 21055:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768008472 with bad export cookie 7576124747667581491 [ 380.034023] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 380.039596] LustreError: 21055:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 380.216292] Lustre: server umount lustre-MDT0001 complete [ 391.541304] Lustre: server umount lustre-OST0000 complete [ 403.643218] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 410.057946] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 410.327241] LustreError: 22888:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 410.379591] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 410.386086] Lustre: Skipped 2 previous similar messages [ 413.035080] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 418.210497] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 420.987222] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 422.713701] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 427.024576] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 430.595618] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 435.579278] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 435.612267] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 435.697827] Lustre: lustre-OST0001: new disk, initializing [ 435.701105] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 437.543355] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 437.548829] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 437.611678] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 439.153058] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 445.669628] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 447.740305] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 461.780122] LustreError: 26367:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 461.783833] LustreError: 26367:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 461.836935] Lustre: server umount lustre-MDT0000 complete [ 463.329658] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 463.336226] LustreError: 23601:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 463.336560] Lustre: Skipped 3 previous similar messages [ 463.345907] LustreError: 23601:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 464.084398] LustreError: 22869:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768008556 with bad export cookie 7576124747667582338 [ 464.086027] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 464.099592] LustreError: 22869:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 464.317296] Lustre: server umount lustre-MDT0001 complete [ 476.700205] Lustre: server umount lustre-OST0000 complete [ 489.471927] Lustre: server umount lustre-OST0001 complete [ 498.139881] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 503.922700] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 504.155254] LustreError: 28144:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 504.167677] LustreError: 28144:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 504.209065] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 504.216623] Lustre: Skipped 3 previous similar messages [ 506.572856] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 511.400903] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 513.857921] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 515.289070] Lustre: 29260:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 518.925823] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 521.995369] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 526.654280] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 529.600214] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 533.992302] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 535.755261] Lustre: 31113:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 545.249155] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 545.255678] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 545.256743] Lustre: Skipped 2 previous similar messages [ 545.262682] Lustre: Skipped 3 previous similar messages [ 549.634680] LustreError: 31316:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 549.637921] LustreError: 31316:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 549.672609] Lustre: server umount lustre-MDT0000 complete [ 550.370892] LustreError: 28144:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 550.377290] LustreError: 28144:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 551.364969] LustreError: 28877:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768008644 with bad export cookie 7576124747667583633 [ 551.367513] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 551.372769] LustreError: 28877:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 551.509332] Lustre: server umount lustre-MDT0001 complete [ 562.707249] Lustre: server umount lustre-OST0000 complete [ 574.845389] Lustre: server umount lustre-OST0001 complete [ 579.213268] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 20:31:10 (1768008670) [ 592.076568] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 596.947605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 599.075452] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 603.127660] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 605.381978] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 606.901375] Lustre: 35011:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 609.984440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 612.603958] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 616.726908] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 619.663580] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 629.038785] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 630.652858] Lustre: 36865:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 637.412931] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 637.415347] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 637.418828] Lustre: Skipped 4 previous similar messages [ 637.421130] Lustre: Skipped 2 previous similar messages [ 643.406989] Lustre: server umount lustre-MDT0000 complete [ 645.190497] LustreError: 34628:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768008737 with bad export cookie 7576124747667584739 [ 645.193578] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 645.196350] LustreError: 34628:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 645.341191] Lustre: server umount lustre-MDT0001 complete [ 656.932567] Lustre: server umount lustre-OST0000 complete [ 668.979935] Lustre: server umount lustre-OST0001 complete [ 672.913121] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 20:32:44 (1768008764) [ 679.962128] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 684.580412] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 684.786837] LustreError: 38991:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 684.798645] LustreError: 38991:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 684.856190] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 684.859496] Lustre: Skipped 7 previous similar messages [ 686.637890] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 690.871575] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 693.325217] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 694.747221] Lustre: 40106:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 697.838917] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 700.564822] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 704.500664] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 707.206233] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 717.067670] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 724.587218] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 724.590441] Lustre: Skipped 1 previous similar message [ 727.009414] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 727.014358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 727.018438] Lustre: Skipped 5 previous similar messages [ 732.374937] LustreError: 42165:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 732.382537] LustreError: 42165:0:(obd_class.h:479:obd_check_dev()) Skipped 51 previous similar messages [ 732.433248] Lustre: server umount lustre-MDT0000 complete [ 734.215233] LustreError: 38970:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768008826 with bad export cookie 7576124747667585845 [ 734.218877] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 734.223786] LustreError: 38970:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 740.467979] Lustre: server umount lustre-MDT0001 complete [ 744.797259] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 758.752828] LustreError: 3629:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff90461a1f8380 x1853890945682816/t0(0) o250->MGC192.168.204.143@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 761.076210] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 765.582069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 767.511783] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 768.874712] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 769.652755] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 770.284155] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 771.710648] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 772.489522] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 776.160979] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 776.165640] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 776.171945] Lustre: Skipped 6 previous similar messages [ 780.545730] Lustre: server umount lustre-MDT0000 complete [ 782.511256] LustreError: 38972:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768008875 with bad export cookie 7576124747667587021 [ 782.513797] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 782.516978] LustreError: 38972:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 782.696910] Lustre: server umount lustre-MDT0001 complete [ 794.834726] Lustre: server umount lustre-OST0000 complete [ 806.846543] Lustre: server umount lustre-OST0001 complete [ 811.370670] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 20:35:02 (1768008902) [ 819.331806] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 825.142825] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 825.404141] LustreError: 46323:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 825.413297] LustreError: 46323:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 15 previous similar messages [ 827.523502] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 831.611691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 833.500576] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 834.790543] Lustre: 47443:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 834.795342] Lustre: 47443:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 837.802636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 840.442028] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 844.606240] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 847.219452] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 856.506517] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 863.901201] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 871.909719] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 871.912288] Lustre: Skipped 18 previous similar messages [ 875.700755] Lustre: server umount lustre-MDT0000 complete [ 877.344328] LustreError: 47061:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768008970 with bad export cookie 7576124747667587805 [ 877.347346] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 877.349824] LustreError: 47061:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 877.473267] Lustre: server umount lustre-MDT0001 complete [ 888.337954] Lustre: server umount lustre-OST0000 complete [ 900.263510] Lustre: server umount lustre-OST0001 complete [ 904.408542] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 20:36:35 (1768008995) [ 905.451687] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 906.250733] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 20:36:37 (1768008997) [ 910.244527] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 913.330585] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 917.183741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 919.712668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 922.566734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 922.591432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 922.714329] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 922.756736] Lustre: lustre-MDT0000: new disk, initializing [ 922.813106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 924.468351] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 928.177227] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 931.085560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 931.117407] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 932.497521] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 932.511152] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 933.654406] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 937.142478] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 937.976583] Lustre: DEBUG MARKER: create test files [ 953.312858] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 955.732450] Lustre: server umount lustre-OST0000 complete [ 957.280655] Lustre: server umount lustre-MDT0000 complete [ 961.471433] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 962.625328] Key type lgssc unregistered [ 962.785952] LNet: 54849:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 962.789489] LNetError: 54849:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 962.799423] LNet: Removed LNI 192.168.204.143@tcp [ 963.184149] Key type .llcrypt unregistered [ 963.185649] Key type ._llcrypt unregistered [ 974.967858] Key type ._llcrypt registered [ 974.969300] Key type .llcrypt registered [ 975.023669] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 975.487628] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 975.501257] alg: No test for adler32 (adler32-zlib) [ 976.376246] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 976.476898] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 978.071167] Key type lgssc registered [ 978.475141] Lustre: Echo OBD driver; http://www.lustre.org/ [ 981.502672] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 981.512131] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 982.704579] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 984.270527] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 986.484362] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 989.147891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 989.283863] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 991.415581] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 992.361871] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 992.398265] Lustre: 57433:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 992.402505] Lustre: 57433:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 992.406233] Lustre: 57433:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 992.410772] Lustre: 57433:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 992.414176] Lustre: 57433:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 992.417216] Lustre: 57433:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 993.417290] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 996.576609] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 998.459095] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1002.976487] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1002.976800] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1002.984194] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1006.518073] LustreError: 58177:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1006.561102] Lustre: server umount lustre-OST0000 complete [ 1008.143934] LustreError: 58379:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1008.148228] LustreError: 58379:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1008.221756] Lustre: server umount lustre-MDT0000 complete [ 1012.075735] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 1013.203936] Key type lgssc unregistered [ 1013.349662] LNet: 58911:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1013.353112] LNetError: 58911:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1013.362486] LNet: Removed LNI 192.168.204.143@tcp [ 1013.685401] Key type .llcrypt unregistered [ 1013.687152] Key type ._llcrypt unregistered [ 1026.291574] Key type ._llcrypt registered [ 1026.292899] Key type .llcrypt registered [ 1026.341195] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1026.731643] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1026.752737] alg: No test for adler32 (adler32-zlib) [ 1027.630346] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 1027.726514] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 1029.311175] Key type lgssc registered [ 1029.740848] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1032.754388] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1032.764521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1033.950547] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1035.448278] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1037.817490] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1040.709980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1040.875703] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1043.301903] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1043.943085] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1043.984697] Lustre: 61589:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1043.989509] Lustre: 61589:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1043.993725] Lustre: 61589:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1043.996900] Lustre: 61589:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1044.000219] Lustre: 61589:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1044.003924] Lustre: 61589:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1045.760241] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1048.962945] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1054.001125] LustreError: 62282:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1054.038599] Lustre: server umount lustre-OST0000 complete [ 1065.955756] LustreError: 62483:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1065.958869] LustreError: 62483:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1066.033312] Lustre: server umount lustre-MDT0000 complete [ 1069.927455] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 1071.180960] Key type lgssc unregistered [ 1071.314510] LNet: 63016:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1071.318764] LNetError: 63016:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1071.328519] LNet: Removed LNI 192.168.204.143@tcp [ 1071.643219] Key type .llcrypt unregistered [ 1071.644687] Key type ._llcrypt unregistered [ 1078.695172] Key type ._llcrypt registered [ 1078.696954] Key type .llcrypt registered [ 1078.742087] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 1085.854914] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1086.231648] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1086.264926] alg: No test for adler32 (adler32-zlib) [ 1087.135823] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 1087.233886] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 1088.839113] Key type lgssc registered [ 1089.328779] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1092.874198] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1095.569717] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1098.128902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1100.710951] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1103.439442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1103.465039] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1103.470677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1104.574125] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1104.588164] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1104.626124] Lustre: lustre-MDT0000: new disk, initializing [ 1104.660191] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1104.669177] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1106.173328] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1110.949565] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1110.975792] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1111.017617] Lustre: 66118:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1111.055227] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1111.057469] Lustre: Skipped 1 previous similar message [ 1111.099154] Lustre: lustre-MDT0001: new disk, initializing [ 1111.137830] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1111.155267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1111.163218] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1112.643597] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1116.276982] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1117.819925] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1120.837460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1120.871248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1120.988508] Lustre: lustre-OST0000: new disk, initializing [ 1120.991664] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1121.032187] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1123.537410] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1127.075601] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1127.428917] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1127.436527] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1127.467660] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1128.753378] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1128.845103] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1130.365397] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1130.459247] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1132.514073] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1132.514678] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1132.523187] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1132.526322] Lustre: Skipped 1 previous similar message [ 1133.295102] LustreError: 69056:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1133.341639] Lustre: server umount lustre-OST0000 complete [ 1136.608290] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1136.611481] LustreError: Skipped 1 previous similar message [ 1136.613507] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1136.617879] Lustre: Skipped 1 previous similar message [ 1136.620423] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1137.632425] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1137.633974] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1141.057863] LustreError: 69257:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1141.060592] LustreError: 69257:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1141.092439] Lustre: server umount lustre-MDT0000 complete [ 1142.555162] LustreError: 66908:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768009235 with bad export cookie 16504203549306347715 [ 1142.557898] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1142.560105] LustreError: 66908:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1142.636566] LustreError: 69458:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1142.639539] LustreError: 69458:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1142.705276] Lustre: server umount lustre-MDT0001 complete [ 1146.354197] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 20:40:37 (1768009237) [ 1150.672090] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1155.737619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1165.140750] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 1168.759736] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1172.309489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1174.807733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1177.466495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1179.939953] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1182.849360] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1182.869480] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1182.955893] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1182.969133] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1183.008912] Lustre: lustre-MDT0000: new disk, initializing [ 1183.041123] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1183.050767] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1184.548573] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1189.475090] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1189.506050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1189.545517] Lustre: 73687:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1189.586894] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1189.589269] Lustre: Skipped 1 previous similar message [ 1189.625768] Lustre: lustre-MDT0001: new disk, initializing [ 1189.661066] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1189.675890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1189.679339] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1191.182867] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1194.619852] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1196.015156] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1198.665567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1198.692129] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1198.800295] Lustre: lustre-OST0000: new disk, initializing [ 1198.802849] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1198.840633] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1200.996094] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1204.185077] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1204.223288] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1204.227475] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1204.260052] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1205.622571] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1205.702749] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1207.078478] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1207.158572] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1209.314677] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1209.315193] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1209.318365] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1209.322208] Lustre: Skipped 1 previous similar message [ 1209.324249] Lustre: Skipped 1 previous similar message [ 1213.871124] LustreError: 76626:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1213.873480] LustreError: 76626:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1213.914908] Lustre: server umount lustre-OST0000 complete [ 1215.455763] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1215.460179] LustreError: Skipped 1 previous similar message [ 1215.462104] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1215.467550] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1215.469985] Lustre: Skipped 1 previous similar message [ 1221.374551] LustreError: 76827:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1221.376627] LustreError: 76827:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1221.410371] Lustre: server umount lustre-MDT0000 complete [ 1222.842802] LustreError: 74466:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768009315 with bad export cookie 16504203549306348597 [ 1222.844678] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1222.849109] LustreError: 74466:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1222.975152] Lustre: server umount lustre-MDT0001 complete [ 1226.349175] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 20:41:58 (1768009318) [ 1228.796069] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1228.960428] LustreError: 77698:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1228.967701] LustreError: 77698:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1228.987163] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1230.443300] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1233.499958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1235.166356] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1237.278674] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1238.613276] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1241.190635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1243.439554] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1245.547720] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1246.988920] Lustre: 80036:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 5530ca7e-a133-4350-8e78-f2901b890d6a at adminstrative request [ 1248.739557] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1248.746212] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1258.985081] Lustre: MGS: Client 5530ca7e-a133-4350-8e78-f2901b890d6a (at 0@lo) reconnecting [ 1258.988733] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xe50ab5af612567a7 to 0xe50ab5af61256a32 [ 1258.994319] Lustre: MGC192.168.204.143@tcp: Connection restored to 0@lo (at 0@lo) [ 1279.740944] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1279.779383] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1279.837473] Lustre: lustre-OST0001: new disk, initializing [ 1279.839747] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1279.868307] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1279.870658] Lustre: Skipped 2 previous similar messages [ 1281.965342] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1285.342677] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1286.357848] LustreError: 81160:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1286.360464] LustreError: 81160:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1286.378295] Lustre: server umount lustre-OST0001 complete [ 1289.198757] LustreError: 79621:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1289.202207] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1289.203202] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1289.203211] Lustre: Skipped 2 previous similar messages [ 1289.211724] Lustre: Skipped 2 previous similar messages [ 1294.306656] LustreError: 79294:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1294.312156] LustreError: 79294:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1299.425736] LustreError: 79621:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1299.430734] LustreError: 79621:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1301.983138] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1302.039897] Lustre: server umount lustre-OST0000 complete [ 1309.629795] LustreError: 81572:0:(obd_class.h:479:obd_check_dev()) Device 24 not setup [ 1309.632422] LustreError: 81572:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1309.662337] Lustre: server umount lustre-MDT0000 complete [ 1309.668439] LustreError: 77698:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1309.675506] LustreError: 77698:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1311.031507] LustreError: 79316:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768009403 with bad export cookie 16504203549306350130 [ 1311.032187] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1311.036270] LustreError: 79316:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1311.151458] Lustre: server umount lustre-MDT0001 complete [ 1314.208264] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 20:43:25 (1768009405) [ 1314.751121] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1315.403955] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 20:43:27 (1768009407) [ 1315.970196] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1316.626274] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 20:43:28 (1768009408) [ 1317.207863] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1317.903109] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 20:43:29 (1768009409) [ 1318.527580] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1319.292713] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 20:43:30 (1768009410) [ 1319.901224] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1321.881023] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1322.333722] loop: module loaded [ 1322.338872] loop0: detected capacity change from 0 to 4294967296 [ 1322.356774] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1322.944225] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1324.123587] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1325.653533] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1328.680150] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1330.030130] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1331.327671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1332.699948] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1335.089295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1335.117524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1335.130440] LustreError: 84439:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1335.133663] LustreError: 84439:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1335.136760] LustreError: 84439:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1335.139840] LustreError: 84439:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1341.460694] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 1344.488422] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1347.686226] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1349.788390] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1351.719108] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1353.764652] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1355.971482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1355.991891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1356.070326] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1356.081104] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1356.116560] Lustre: lustre-MDT0000: new disk, initializing [ 1356.140915] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1356.147786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1357.490933] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1361.686888] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1361.708031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1361.737037] Lustre: 87275:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1361.827835] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1363.073242] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1366.062419] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1367.201433] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1369.758454] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1369.787275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1371.795182] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1372.596623] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1372.629869] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1374.812337] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1376.104635] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1376.171679] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1377.412276] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1377.472152] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1380.141635] LustreError: 90216:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1380.143736] LustreError: 90216:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1380.177915] Lustre: server umount lustre-OST0000 complete [ 1382.367758] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1382.370850] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1382.374345] Lustre: Skipped 2 previous similar messages [ 1382.375867] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1382.377757] Lustre: Skipped 7 previous similar messages [ 1392.125881] Lustre: server umount lustre-MDT0000 complete [ 1393.120143] LustreError: 87287:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1393.125845] LustreError: 87287:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1393.534173] LustreError: 87266:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768009486 with bad export cookie 16504203549306350690 [ 1393.537420] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1393.538973] LustreError: 87266:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1403.930289] Lustre: server umount lustre-MDT0001 complete [ 1407.025656] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 20:44:58 (1768009498) [ 1409.278313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1409.435853] LustreError: 91290:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1410.740751] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1413.764201] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1415.265042] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1417.217937] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1418.529854] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1420.886023] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1422.950772] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1424.903486] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1426.321817] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1434.539082] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 8 sec [ 1435.809194] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1435.883515] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1438.410578] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1438.441365] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1438.508027] Lustre: lustre-OST0001: new disk, initializing [ 1438.509907] Lustre: Skipped 2 previous similar messages [ 1438.513383] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1438.519550] Lustre: Skipped 3 previous similar messages [ 1440.701813] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1443.937605] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1445.344731] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1446.921675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1446.925456] Lustre: Skipped 2 previous similar messages [ 1446.928616] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1446.956548] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1447.462369] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1448.988441] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1449.069390] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1459.140699] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1465.988437] Lustre: Failing over lustre-MDT0000 [ 1466.380884] LustreError: 96243:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1466.383055] LustreError: 96243:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1466.412886] Lustre: server umount lustre-MDT0000 complete [ 1467.362872] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1467.363949] LustreError: 96359:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1467.367714] Lustre: Skipped 4 previous similar messages [ 1467.372883] LustreError: 96359:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1481.493315] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1481.494932] LDISKFS-fs (dm-0): recovery complete [ 1481.498452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1481.541214] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1481.663239] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1482.996385] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1486.334705] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1486.335270] Lustre: 91314:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1486.826362] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1486.836150] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1486.841022] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 3 previous similar messages [ 1486.852031] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1487.167124] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1487.481392] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1487.485147] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1487.799117] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1487.801559] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1488.121407] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1488.124937] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 6 previous similar messages [ 1488.753478] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1488.757081] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1489.071108] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1489.073743] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1490.327757] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1490.331378] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 6 previous similar messages [ 1490.959891] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1490.963340] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1491.279138] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1491.281835] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1494.409036] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1494.413371] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 1495.041409] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1495.044662] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1495.359117] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1495.361705] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1502.546061] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1502.553596] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 25 previous similar messages [ 1503.185076] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1503.188131] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1503.503057] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1503.506783] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1518.801340] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1518.804689] Lustre: 96989:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 51 previous similar messages [ 1519.433712] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1519.436627] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1519.751150] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1519.754600] LustreError: 96989:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1546.287163] Lustre: 96989:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1546.290837] Lustre: 96989:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1546.294075] Lustre: 96989:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 4d249483-59e6-4b6a-b0b3-1d57e3fa05f7@192.168.204.43@tcp [ 1546.297984] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1546.300555] Lustre: 96989:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@000000001146cd18 x1853891985609856/t0(8589934787) o101->4d249483-59e6-4b6a-b0b3-1d57e3fa05f7@192.168.204.43@tcp:479/0 lens 592/0 e 0 to 0 dl 1768009649 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1546.310900] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1546.317197] Lustre: lustre-MDT0000: Denying connection for new client 4d249483-59e6-4b6a-b0b3-1d57e3fa05f7 (at 192.168.204.43@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1546.318837] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1546.339601] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1546.342191] Lustre: Skipped 2 previous similar messages [ 1546.343142] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1546.362131] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 1546.362163] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:82 to 0x280000401:577) [ 1556.960969] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1556.963605] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1556.968877] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1556.971321] Lustre: Skipped 3 previous similar messages [ 1561.149274] Lustre: server umount lustre-OST0000 complete [ 1563.617814] LustreError: 93376:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1563.623023] LustreError: 93376:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 15 previous similar messages [ 1568.590482] Lustre: server umount lustre-OST0001 complete [ 1576.027736] Lustre: server umount lustre-MDT0000 complete [ 1577.227176] LustreError: 91269:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768009669 with bad export cookie 16504203549306401888 [ 1577.230571] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1577.230913] LustreError: 91269:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1577.357124] Lustre: server umount lustre-MDT0001 complete [ 1580.187616] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 20:47:51 (1768009671) [ 1582.843898] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1582.847873] Lustre: *** cfs_fail_loc=197, val=0*** [ 1598.303679] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1603.423388] LustreError: 98943:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.143@tcp: failed processing log, type 4: rc = -110 [ 1608.551183] Lustre: *** cfs_fail_loc=197, val=0*** [ 1608.552728] Lustre: Skipped 65 previous similar messages [ 1629.023302] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1629.025489] Lustre: Skipped 7 previous similar messages [ 1630.797763] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1632.590326] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1633.245950] LustreError: 99607:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 1633.248796] LustreError: 99607:0:(obd_class.h:479:obd_check_dev()) Skipped 33 previous similar messages [ 1633.282970] Lustre: server umount lustre-OST0000 complete [ 1635.975560] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 20:48:47 (1768009727) [ 1649.706194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1651.300797] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1654.730994] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1656.777996] LustreError: 100484:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1656.782064] LustreError: 100484:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1656.785419] LustreError: 100484:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1656.908447] Lustre: server umount lustre-MDT0000 complete [ 1662.558092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1669.578938] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 1673.143690] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1676.750571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1679.095252] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1681.425443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1683.806321] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1686.319517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1686.340867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1686.424514] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1686.439868] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1686.476520] Lustre: lustre-MDT0000: new disk, initializing [ 1686.504428] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1687.854231] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1692.229605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1692.257643] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1692.296748] Lustre: 104520:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1692.405413] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1693.898664] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1697.296977] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1698.497309] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1700.832129] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1700.860491] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1702.997629] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1706.458349] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1707.953315] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1709.591782] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1710.079593] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1711.504176] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1711.581844] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1714.657575] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1714.658211] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1714.660368] LustreError: Skipped 3 previous similar messages [ 1714.664699] Lustre: Skipped 5 previous similar messages [ 1714.665472] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1714.669370] Lustre: Skipped 9 previous similar messages [ 1718.353470] Lustre: server umount lustre-OST0000 complete [ 1734.111114] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1734.174716] Lustre: server umount lustre-MDT0000 complete [ 1735.136949] LustreError: 105315:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1735.142866] LustreError: 105315:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1735.498306] LustreError: 104512:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768009828 with bad export cookie 16504203549306412619 [ 1735.501102] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1735.502574] LustreError: 104512:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1735.621463] Lustre: server umount lustre-MDT0001 complete [ 1738.717327] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 20:50:30 (1768009830) [ 1741.756898] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 1742.802740] Key type lgssc unregistered [ 1742.937376] LNet: 108601:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1742.940249] LNetError: 108601:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1742.950322] LNet: Removed LNI 192.168.204.143@tcp [ 1743.260146] Key type .llcrypt unregistered [ 1743.261386] Key type ._llcrypt unregistered [ 1749.991709] Key type ._llcrypt registered [ 1749.992692] Key type .llcrypt registered [ 1750.035345] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 1756.499868] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1756.895578] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1756.921621] alg: No test for adler32 (adler32-zlib) [ 1757.784258] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 1757.876906] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 1759.463156] Key type lgssc registered [ 1759.867390] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1763.062793] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1765.233554] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1767.267882] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1769.698727] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1771.700319] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1774.268839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1774.288980] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1774.296416] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1775.389625] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1775.402953] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1775.441028] Lustre: lustre-MDT0000: new disk, initializing [ 1775.468631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1775.475574] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1776.863620] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1781.040195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1781.067751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1781.185752] Lustre: lustre-OST0000: new disk, initializing [ 1781.188065] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1781.189408] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 1781.189921] Lustre: Skipped 1 previous similar message [ 1781.214864] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1783.122116] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1786.354298] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1786.357722] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1786.366074] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1787.234792] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1787.258836] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1787.315992] Lustre: lustre-OST0001: new disk, initializing [ 1787.319023] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1787.348959] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1789.193540] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1792.496099] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1792.500352] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1792.509032] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1797.600900] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1797.601568] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1797.605991] Lustre: Skipped 1 previous similar message [ 1797.610928] Lustre: Skipped 1 previous similar message [ 1799.290806] LustreError: 114157:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 1799.339381] Lustre: server umount lustre-MDT0000 complete [ 1801.547752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1803.058142] LustreError: 111911:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768009895 with bad export cookie 14490397395493755514 [ 1803.060747] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1803.062562] LustreError: 111911:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1803.084075] LustreError: 114752:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1803.086261] LustreError: 114752:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1803.101471] Lustre: server umount lustre-OST0000 complete [ 1804.262886] LustreError: 114952:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 1804.264865] LustreError: 114952:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1804.304825] Lustre: server umount lustre-OST0001 complete [ 1806.943871] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 20:51:38 (1768009898) [ 1809.809543] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1812.685914] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1813.064648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1814.846487] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1815.179837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1817.254644] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 20:51:49 (1768009909) [ 1821.630755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1822.273036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1822.945253] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1823.582933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1823.896136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1824.511332] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1829.798715] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 1832.923376] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1836.054894] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1838.214690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1840.236783] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1842.285898] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1844.418691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1844.439783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1844.513367] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1844.522966] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1844.553043] Lustre: lustre-MDT0000: new disk, initializing [ 1844.572967] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1844.578513] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1845.740346] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1850.054725] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1850.081640] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1850.115438] Lustre: 120262:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1850.137857] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1850.140194] Lustre: Skipped 1 previous similar message [ 1850.175519] Lustre: lustre-MDT0001: new disk, initializing [ 1850.205872] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1850.218295] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1850.222802] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1851.631410] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1854.666426] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1855.870068] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1858.020329] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1858.042831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1860.001059] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1860.408381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1860.411677] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1860.440566] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1863.143927] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1864.473247] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1864.547988] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1865.799067] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1865.868414] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1870.817941] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1870.818376] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1870.819990] LustreError: Skipped 1 previous similar message [ 1870.820589] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1870.823276] Lustre: Skipped 1 previous similar message [ 1870.827437] Lustre: Skipped 1 previous similar message [ 1872.557212] LustreError: 123199:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1872.558947] LustreError: 123199:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1872.592045] Lustre: server umount lustre-OST0000 complete [ 1875.936746] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1875.937172] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1875.944150] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1875.946268] Lustre: Skipped 1 previous similar message [ 1881.056337] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1881.058203] Lustre: Skipped 1 previous similar message [ 1888.223137] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1888.255834] LustreError: 123400:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1888.258893] LustreError: 123400:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1888.295058] Lustre: server umount lustre-MDT0000 complete [ 1889.557066] LustreError: 120253:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768009982 with bad export cookie 14490397395493756732 [ 1889.560242] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1889.560543] LustreError: 120253:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1889.683871] Lustre: server umount lustre-MDT0001 complete [ 1892.701379] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 20:53:04 (1768009984) [ 1895.047056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1895.227576] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1895.230366] Lustre: Skipped 1 previous similar message [ 1896.613054] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1899.345293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1900.686656] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1902.375099] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1903.502435] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1905.703130] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1907.600672] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1909.275115] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1927.072768] Lustre: *** cfs_fail_loc=159, val=0*** [ 1943.562879] Lustre: lustre-MDT0000: Client a1edefc9-c9e9-45f6-b68b-99010bf8622d (at 192.168.204.43@tcp) reconnecting [ 1944.035837] Lustre: *** cfs_fail_loc=159, val=0*** [ 1944.037734] Lustre: Skipped 6 previous similar messages [ 1960.457891] Lustre: lustre-MDT0000: Client a1edefc9-c9e9-45f6-b68b-99010bf8622d (at 192.168.204.43@tcp) reconnecting [ 1966.560413] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1966.561454] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1966.562574] LustreError: Skipped 1 previous similar message [ 1966.562792] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1966.562796] Lustre: Skipped 2 previous similar messages [ 1966.566734] Lustre: Skipped 2 previous similar messages [ 1967.598084] LustreError: 126784:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1967.600181] LustreError: 126784:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1967.630273] Lustre: server umount lustre-OST0000 complete [ 1971.167903] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1971.170328] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1975.004289] Lustre: server umount lustre-MDT0000 complete [ 1976.246435] LustreError: 126685:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768010068 with bad export cookie 14490397395493757600 [ 1976.249254] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1976.249897] LustreError: 126685:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1976.367942] Lustre: server umount lustre-MDT0001 complete [ 1979.819391] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 1980.870628] Key type lgssc unregistered [ 1980.997418] LNet: 127719:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1981.001524] LNetError: 127719:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1981.013315] LNet: Removed LNI 192.168.204.143@tcp [ 1981.322145] Key type .llcrypt unregistered [ 1981.323409] Key type ._llcrypt unregistered [ 1990.346952] Key type ._llcrypt registered [ 1990.347924] Key type .llcrypt registered [ 1990.381198] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1990.709270] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1990.738120] alg: No test for adler32 (adler32-zlib) [ 1991.580431] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 1991.650975] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 1993.223119] Key type lgssc registered [ 1993.561150] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1996.064782] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1996.072655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1997.231844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1998.494500] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2001.173619] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2001.281287] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2002.444517] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2004.061384] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2005.110560] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2007.234164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2007.339321] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2009.111444] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2010.743239] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2021.672191] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2030.918516] Lustre: *** cfs_fail_loc=159, val=0*** [ 2047.497252] Lustre: lustre-MDT0000: Client b395ee94-f64d-46b3-811f-c27c1cc9603d (at 192.168.204.43@tcp) reconnecting [ 2048.246648] Lustre: *** cfs_fail_loc=159, val=0*** [ 2064.395897] Lustre: lustre-MDT0001: Client b395ee94-f64d-46b3-811f-c27c1cc9603d (at 192.168.204.43@tcp) reconnecting [ 2064.888390] Lustre: *** cfs_fail_loc=159, val=0*** [ 2064.889565] Lustre: Skipped 3 previous similar messages [ 2081.289523] Lustre: lustre-MDT0001: Client b395ee94-f64d-46b3-811f-c27c1cc9603d (at 192.168.204.43@tcp) reconnecting [ 2082.367437] Lustre: 131826:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2083.175842] Lustre: *** cfs_fail_loc=159, val=0*** [ 2083.177760] Lustre: Skipped 3 previous similar messages [ 2099.726386] Lustre: lustre-MDT0000: Client d94c873d-504c-4fb0-a42c-a5d8ec0677da (at 192.168.204.43@tcp) reconnecting [ 2100.329476] Lustre: *** cfs_fail_loc=159, val=0*** [ 2100.331362] Lustre: Skipped 14 previous similar messages [ 2116.622792] Lustre: lustre-MDT0000: Client d94c873d-504c-4fb0-a42c-a5d8ec0677da (at 192.168.204.43@tcp) reconnecting [ 2117.175294] Lustre: 132117:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2134.498439] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2134.499425] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2134.499958] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2134.502686] Lustre: Skipped 1 previous similar message [ 2139.617766] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2139.620055] Lustre: Skipped 2 previous similar messages [ 2140.204111] LustreError: 132240:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2140.243500] Lustre: server umount lustre-OST0000 complete [ 2144.736913] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2144.737811] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2144.738175] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2144.741477] Lustre: Skipped 1 previous similar message [ 2147.712111] LustreError: 132441:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2147.714101] LustreError: 132441:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2147.739222] Lustre: server umount lustre-MDT0000 complete [ 2149.032178] LustreError: 131946:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768010241 with bad export cookie 549915696709934284 [ 2149.035057] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2149.036408] LustreError: 131946:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2149.110636] LustreError: 132643:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2149.113128] LustreError: 132643:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2149.174703] Lustre: server umount lustre-MDT0001 complete [ 2152.812526] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 2153.890790] Key type lgssc unregistered [ 2154.023415] LNet: 133174:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2154.025763] LNetError: 133174:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2154.037312] LNet: Removed LNI 192.168.204.143@tcp [ 2154.349917] Key type .llcrypt unregistered [ 2154.351057] Key type ._llcrypt unregistered [ 2163.665930] Key type ._llcrypt registered [ 2163.667157] Key type .llcrypt registered [ 2163.707977] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 2164.165840] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2164.173605] alg: No test for adler32 (adler32-zlib) [ 2165.033030] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 2165.121557] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 2166.711266] Key type lgssc registered [ 2167.095061] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2169.747700] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2169.754748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2170.912469] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2172.188721] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2175.001609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2175.124322] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2176.433288] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2178.162262] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2179.331827] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2181.536540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2181.603181] LustreError: 136146:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2181.609675] LustreError: 136146:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2181.653200] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2181.912776] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2183.596417] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2185.004948] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2185.462603] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2188.640609] Lustre: 136794:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2189.308323] Lustre: 136842:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2189.805925] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2190.979380] Lustre: 136989:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2191.301065] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2191.305019] Lustre: Skipped 1 previous similar message [ 2191.615033] Lustre: 137087:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2195.425843] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2195.426411] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2195.428283] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2195.431732] Lustre: Skipped 1 previous similar message [ 2198.637171] LustreError: 137185:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2198.670494] Lustre: server umount lustre-OST0000 complete [ 2200.544818] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2200.545487] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2200.548330] Lustre: Skipped 1 previous similar message [ 2200.552052] Lustre: Skipped 2 previous similar messages [ 2205.664066] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2205.665988] Lustre: Skipped 1 previous similar message [ 2210.784279] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2210.786953] Lustre: Skipped 1 previous similar message [ 2214.367116] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2214.400119] LustreError: 137394:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2214.401860] LustreError: 137394:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2214.428052] Lustre: server umount lustre-MDT0000 complete [ 2215.639915] LustreError: 136952:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768010308 with bad export cookie 8658781251179836263 [ 2215.643135] LustreError: 136952:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2215.644227] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2215.716781] LustreError: 137597:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2215.719669] LustreError: 137597:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2215.766510] Lustre: server umount lustre-MDT0001 complete [ 2219.432824] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 2220.563876] Key type lgssc unregistered [ 2220.710659] LNet: 138128:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2220.714389] LNetError: 138128:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2220.728340] LNet: Removed LNI 192.168.204.143@tcp [ 2221.040580] Key type .llcrypt unregistered [ 2221.041943] Key type ._llcrypt unregistered [ 2229.762847] Key type ._llcrypt registered [ 2229.764178] Key type .llcrypt registered [ 2229.806955] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 2230.174560] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2230.235851] alg: No test for adler32 (adler32-zlib) [ 2231.092463] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 2231.184255] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 2232.767161] Key type lgssc registered [ 2233.147151] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2235.811981] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2235.818847] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2236.979479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2238.241087] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2241.025022] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2241.154656] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2242.450151] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2244.237300] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2245.436736] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2247.675357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2247.786065] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2249.583321] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2250.853427] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2251.267123] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2262.122113] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2269.257121] Lustre: *** cfs_fail_loc=159, val=0*** [ 2285.566455] Lustre: lustre-MDT0000: Client 2656cfa8-244f-4dee-8f20-c32ec02ca2db (at 192.168.204.43@tcp) reconnecting [ 2302.432028] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2302.434853] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2302.439030] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2303.457151] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2303.458845] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2303.458994] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2303.459634] Lustre: Skipped 1 previous similar message [ 2306.477625] LustreError: 141905:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2306.510541] Lustre: server umount lustre-OST0000 complete [ 2308.063802] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2308.066086] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2308.070232] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2313.695872] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2313.697713] Lustre: Skipped 2 previous similar messages [ 2313.787434] LustreError: 142107:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2313.789449] LustreError: 142107:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2313.816962] Lustre: server umount lustre-MDT0000 complete [ 2314.970213] LustreError: 139479:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768010407 with bad export cookie 11195684148169843398 [ 2314.972665] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2314.973659] LustreError: 139479:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2315.030489] LustreError: 142308:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2315.032806] LustreError: 142308:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2315.080529] Lustre: server umount lustre-MDT0001 complete [ 2318.208923] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 2319.126174] Key type lgssc unregistered [ 2319.241336] LNet: 142839:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2319.244064] LNetError: 142839:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2319.252245] LNet: Removed LNI 192.168.204.143@tcp [ 2319.531073] Key type .llcrypt unregistered [ 2319.532148] Key type ._llcrypt unregistered [ 2328.397069] Key type ._llcrypt registered [ 2328.398589] Key type .llcrypt registered [ 2328.439084] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 2328.781084] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2328.797882] alg: No test for adler32 (adler32-zlib) [ 2329.673259] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 2329.769263] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 2331.359138] Key type lgssc registered [ 2331.769272] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2334.462353] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2334.468746] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2335.613363] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2336.917375] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2339.582393] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2339.693466] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2340.956210] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2342.684245] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2343.892959] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2346.169831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2346.296811] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2348.231429] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2349.350586] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2350.030429] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2360.616463] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2378.203290] Lustre: lustre-OST0000: evicting bd41ff7f-68fb-4983-aaf1-2cd40b6ed639 (at 192.168.204.43@tcp) by administrative request [ 2378.205726] Lustre: Skipped 1 previous similar message [ 2405.855955] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2405.858374] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2405.863435] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2406.880902] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2406.882696] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2406.882797] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2406.891254] Lustre: Skipped 1 previous similar message [ 2408.431193] LustreError: 147029:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2408.470570] Lustre: server umount lustre-OST0000 complete [ 2411.487827] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2411.491561] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2411.496827] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2417.120274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2417.122342] Lustre: Skipped 2 previous similar messages [ 2422.240319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2422.242767] Lustre: Skipped 1 previous similar message [ 2424.287179] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2424.329879] LustreError: 147231:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2424.331875] LustreError: 147231:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2424.363859] Lustre: server umount lustre-MDT0000 complete [ 2425.629192] LustreError: 144939:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768010518 with bad export cookie 13695943625631759190 [ 2425.631726] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2425.633129] LustreError: 144939:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2425.697986] LustreError: 147434:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2425.700616] LustreError: 147434:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2425.754320] Lustre: server umount lustre-MDT0001 complete [ 2429.334399] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 2430.363433] Key type lgssc unregistered [ 2430.492348] LNet: 147964:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2430.495579] LNetError: 147964:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2430.501356] LNet: Removed LNI 192.168.204.143@tcp [ 2430.792325] Key type .llcrypt unregistered [ 2430.793350] Key type ._llcrypt unregistered [ 2454.788149] Key type ._llcrypt registered [ 2454.790941] Key type .llcrypt registered [ 2454.836671] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 2455.163408] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2455.176677] alg: No test for adler32 (adler32-zlib) [ 2456.054596] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 2456.148197] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 2457.743177] Key type lgssc registered [ 2458.121400] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2460.773486] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2460.779484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2461.939703] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2463.246189] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2466.191307] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2466.347397] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2467.725225] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2469.665950] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2470.841490] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2473.116545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2473.257516] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2475.234648] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2476.327221] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2477.104482] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2487.591125] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2491.872376] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2491.875017] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2491.879611] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2492.898238] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2492.898550] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2492.899335] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2496.556170] LustreError: 152682:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2496.592456] Lustre: server umount lustre-OST0000 complete [ 2498.016926] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2498.017561] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2498.023322] Lustre: Skipped 1 previous similar message [ 2498.025069] Lustre: Skipped 1 previous similar message [ 2503.136794] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2503.139454] Lustre: Skipped 2 previous similar messages [ 2508.256139] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2508.258261] Lustre: Skipped 1 previous similar message [ 2512.351098] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2512.385612] LustreError: 152884:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2512.387743] LustreError: 152884:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2512.419817] Lustre: server umount lustre-MDT0000 complete [ 2513.376975] LustreError: 150405:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2513.383031] LustreError: 150405:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2513.672049] LustreError: 150386:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768010606 with bad export cookie 2155568232600434069 [ 2513.675386] LustreError: 150386:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2513.675506] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2513.745340] LustreError: 153086:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2513.748449] LustreError: 153086:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2513.801536] Lustre: server umount lustre-MDT0001 complete [ 2517.523392] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 2518.638753] Key type lgssc unregistered [ 2518.775396] LNet: 153617:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2518.777915] LNetError: 153617:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2518.786435] LNet: Removed LNI 192.168.204.143@tcp [ 2519.108874] Key type .llcrypt unregistered [ 2519.110126] Key type ._llcrypt unregistered [ 2528.143919] Key type ._llcrypt registered [ 2528.145049] Key type .llcrypt registered [ 2528.190975] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 2528.544668] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2528.569110] alg: No test for adler32 (adler32-zlib) [ 2529.441320] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 2529.532241] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 2531.111206] Key type lgssc registered [ 2531.508392] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2534.035408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2536.558583] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 21:03:48 (1768010628) [ 2542.080500] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 2545.298958] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 2548.871238] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2551.000125] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2553.040808] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2555.262911] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2557.653370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2557.669843] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2557.675177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2558.765184] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2558.776239] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2558.813504] Lustre: lustre-MDT0000: new disk, initializing [ 2558.837243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2558.844255] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2560.218784] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2564.481373] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2564.511698] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2564.552369] Lustre: 157809:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2564.582437] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2564.584701] Lustre: Skipped 1 previous similar message [ 2564.622983] Lustre: lustre-MDT0001: new disk, initializing [ 2564.663328] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2564.676635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2564.680708] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2566.046504] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2569.209477] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2570.485837] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2572.905948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2572.927665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2573.035305] Lustre: lustre-OST0000: new disk, initializing [ 2573.037690] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2573.069110] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2575.087294] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2578.258877] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2578.434434] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2578.438077] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2578.478625] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2588.641629] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2588.643031] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2588.644136] LustreError: Skipped 1 previous similar message [ 2588.644180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2588.649538] Lustre: Skipped 1 previous similar message [ 2592.749660] LustreError: 160395:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2592.784201] Lustre: server umount lustre-OST0000 complete [ 2595.295888] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2595.298368] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2595.302617] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2595.304514] Lustre: Skipped 1 previous similar message [ 2598.880158] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2598.880766] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2600.126303] LustreError: 160597:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2600.128448] LustreError: 160597:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2600.151315] Lustre: server umount lustre-MDT0000 complete [ 2601.401361] LustreError: 160598:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768010694 with bad export cookie 280074944588118507 [ 2601.406827] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2601.472096] LustreError: 160799:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2601.473586] LustreError: 160799:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2601.529797] Lustre: server umount lustre-MDT0001 complete [ 2605.047894] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 2606.112867] Key type lgssc unregistered [ 2606.238387] LNet: 161330:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2606.241312] LNetError: 161330:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2606.253354] LNet: Removed LNI 192.168.204.143@tcp [ 2606.553949] Key type .llcrypt unregistered [ 2606.555161] Key type ._llcrypt unregistered [ 2615.545686] Key type ._llcrypt registered [ 2615.546991] Key type .llcrypt registered [ 2615.596075] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 2615.969826] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2615.992872] alg: No test for adler32 (adler32-zlib) [ 2616.860141] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 2616.956288] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 2618.543176] Key type lgssc registered [ 2618.912204] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2621.735667] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2621.746479] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2622.916304] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2624.263025] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2627.293033] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2627.422235] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2628.777479] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2630.580404] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2631.824592] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2634.201246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2634.325882] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2636.299366] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2638.145765] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2647.654642] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2650.836574] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2650.843088] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2650.847437] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2650.850597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.854127] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2651.178687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.534988] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2651.537360] Lustre: Skipped 1 previous similar message [ 2651.539577] LustreError: 162723:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2651.543520] LustreError: 162723:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2651.545801] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2651.551160] Lustre: Skipped 1 previous similar message [ 2651.552602] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2651.554446] Lustre: Skipped 1 previous similar message [ 2651.556231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.560229] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2651.562650] Lustre: Skipped 1 previous similar message [ 2651.868957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.189239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.523763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.863848] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2652.867308] Lustre: Skipped 3 previous similar messages [ 2652.873938] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2652.878896] Lustre: Skipped 3 previous similar messages [ 2652.880390] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2652.882455] Lustre: Skipped 3 previous similar messages [ 2652.883846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.888398] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2652.892318] Lustre: Skipped 3 previous similar messages [ 2653.205974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.543628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.863182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.213288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.516729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.829708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.133200] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2655.135661] Lustre: Skipped 6 previous similar messages [ 2655.141319] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2655.144559] Lustre: Skipped 6 previous similar messages [ 2655.145919] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2655.148115] Lustre: Skipped 6 previous similar messages [ 2655.149669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.152866] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2655.155633] Lustre: Skipped 6 previous similar messages [ 2655.467092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.767197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.058740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.361863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.676425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.991679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.321408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.643047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.976558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.275493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.589242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.909758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.235805] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2659.238049] Lustre: Skipped 12 previous similar messages [ 2659.243968] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2659.247349] Lustre: Skipped 12 previous similar messages [ 2659.248931] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2659.251446] Lustre: Skipped 12 previous similar messages [ 2659.253236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.256488] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2659.258903] Lustre: Skipped 12 previous similar messages [ 2659.572611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.882190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.192327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.519803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.827874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.130141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.423858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.729953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.035960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.347769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.659326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.978083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.287675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.597682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.896514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.204570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.508963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.816522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.122549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.446673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.752108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.058210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.345215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.664633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.960329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.255602] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2667.257566] Lustre: Skipped 25 previous similar messages [ 2667.263813] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2667.268905] Lustre: Skipped 25 previous similar messages [ 2667.271144] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2667.274274] Lustre: Skipped 25 previous similar messages [ 2667.275554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.279611] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2667.281843] Lustre: Skipped 25 previous similar messages [ 2667.580839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.917653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.224643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.539750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.846954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.164807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.467912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.775673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.092046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.413892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.726333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.034620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.338455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.661501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.964902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.267343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.578714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.899316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.188467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.471038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.752299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.045255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.333729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.627028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.916804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.216296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.515258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.803499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.100647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.420503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.720554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.005339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.324873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.652410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.953638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.249682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.547784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.874699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.179642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.483738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.789924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.117331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.438521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.762339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.079390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.403706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.737423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.069660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.390841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.713649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.013774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.326388] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2683.328697] Lustre: Skipped 51 previous similar messages [ 2683.334993] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2683.339247] Lustre: Skipped 51 previous similar messages [ 2683.341407] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2683.343984] Lustre: Skipped 51 previous similar messages [ 2683.346144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.351971] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2683.353882] Lustre: Skipped 51 previous similar messages [ 2683.667072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.994316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.305251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.612825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.934619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.285591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.600810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.913197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.240177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.574828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.891929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.200666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.506949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.826827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.121249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.434706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.743414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.081307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.392460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.700142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.004697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.335838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.657022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.969374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.266643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.589049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.903385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.204702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.494396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.795627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.095763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.408822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.719800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.042338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.344133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.658816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.951233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.247430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.533912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.838822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.150990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.475275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.779887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.084060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.385668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.714856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.015509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.329398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.628232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.943480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.240724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.544465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.844262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.170851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.462474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.755914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.063549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.371844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.675833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.967704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.258699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.571419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.871432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.169982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.464946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.786843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.091069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.384877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.681548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.988276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.274599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.568691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.868378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.182484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.482697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.785711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.083703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.388070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.678090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.970148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.265248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.584098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.885365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.189750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.499883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.820828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.119783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.414697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.729399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.040027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.331874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.630042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.913680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.218166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.504444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.806491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.111592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.410350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.703779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.014490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.309356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.633778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.938034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.241871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.537268] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2715.539727] Lustre: Skipped 104 previous similar messages [ 2715.545206] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2715.550254] Lustre: Skipped 104 previous similar messages [ 2715.551862] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2715.553913] Lustre: Skipped 104 previous similar messages [ 2715.555410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.558855] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2715.560581] Lustre: Skipped 104 previous similar messages [ 2715.895350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.202739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.497986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.793026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.098355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.393741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.694945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.993683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.312707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.599891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.894868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.177034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.479704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.759635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.039147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.326751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.633342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.957214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.254187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.559385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.889537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.179598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.475446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.783604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.100881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.394375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.681843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.987951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.293488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.593560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.899612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.184361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.491932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.811101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.117034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.413719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.721421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.003431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.300836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.597598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.919953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.220917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.517153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.812336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.106213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.393874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.685710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.990937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.303620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.611376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.909620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.208707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.505296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.798017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.081917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.374016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.667263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.954904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.246042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.536202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.861589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.170177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.458960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.743989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.050496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.347413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.647440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.949448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.263513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.562466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.870292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.160432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.469763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.768426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.073208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.375557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.687385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.985961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.278393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.570576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.885401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.182411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.481552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.773577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.072634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.364742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.647341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.931786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.233073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.523683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.834634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.137196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.436979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.722599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.028705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.330609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.641293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.936934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.234850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.529058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.851024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.137662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.422232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.735809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.064936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.391923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.697088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.007482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.319870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.626635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.949570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.257048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.595352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.910064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.219823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.530416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.847234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.148823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.437713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.725691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.039367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.329449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.614266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.897704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.193327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.486348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.778064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.081823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.404752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.723857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.057618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.361880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.672924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.963437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.246187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.535917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.838072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.127762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.435547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.757365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.098323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.402266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.701806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.032733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.388037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.712282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.033272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.357023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.700874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.017214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.344234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.648346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.960365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.262533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.563902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.864031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.176033] LustreError: 162723:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2763.179493] LustreError: 162723:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2763.180034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.481925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.785297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.089690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.424079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.742524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.045439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.350792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.686292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.996045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.296501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.590026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.897963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.212799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.523996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.847338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.189885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.489623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.796199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.119349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.445331] LustreError: 162723:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2769.449259] LustreError: 162723:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2769.450210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.767844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.087714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.399579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.739870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.054349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.354592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.653705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.004035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.326958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.640303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.965112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.305363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.619751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.957977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.289359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.629989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.948635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.269418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.585324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.900239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.193928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.496702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.804071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.130873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.437051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.751962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.070087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.394511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.726052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.036507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.343270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.685556] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2779.687819] Lustre: Skipped 208 previous similar messages [ 2779.695949] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2779.702583] Lustre: Skipped 208 previous similar messages [ 2779.705264] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2779.707583] Lustre: Skipped 208 previous similar messages [ 2779.708861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.712578] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2779.715340] Lustre: Skipped 208 previous similar messages [ 2780.028486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.358980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.674180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.043557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.367166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.697947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.017422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.345315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.666426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.999351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.323235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.675242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.006760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.333873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.658315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.013165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.345185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.650547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.956034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.273881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.576351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.871123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.177882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.499806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.801979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.107857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.416745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.739831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.064350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.393681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.707507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.041158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.349610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.662168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.990367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.306781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.616878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.923601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.231573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.545756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.850342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.182213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.520901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.838738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.149469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.476281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.792976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.165856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.483858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.805252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.114920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.480249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.795711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.089252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.386079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.720104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.031067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.340397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.662463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.021707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.323384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.634608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.960492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.299091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.616933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.923608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.221251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.559906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.885231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.201714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.522282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.876968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.195319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.511959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.823948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.133675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.438212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.762754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.093877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.425261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.728475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.055962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.366263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.690772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.000034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.296824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.612334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.951683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.281501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.590550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.888036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.228674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.544151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.849115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.167118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.489028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.794229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.091645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.387862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.714440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.020948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.338117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.637648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.957942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.267717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.584076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.896771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.216093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.517047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.818681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.111468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.426202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.706246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.007627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.319422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.644467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.934256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.232639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.521193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.829174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.131598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.454368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.774779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.119244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.423137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.714813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.026845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.345398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.650903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.962758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.280834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.595259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.894178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.191946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.515868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.846875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.171193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.481317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.807529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.163454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.469181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.778563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.081661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.398461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.711037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.019383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.329874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.654632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.972876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.270708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.575561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.895946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.198019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.502673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.801967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.132394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.433220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.737928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.029059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.346534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.625970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.921544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.228665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.547643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.853206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.154223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.462506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.776718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.073505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.384446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.688720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.010306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.323837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.640291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.954983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.289261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.589488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.896419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.191740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.503751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.805439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.134705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.439022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.752272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.046395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.340475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.654345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.975344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.282168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.591898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.893231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.201505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.496981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.792321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.095321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.397051] LustreError: 162723:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2841.399732] LustreError: 162723:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2841.400607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.697407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.007799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.306389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.647769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.955824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.272382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.570983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.886126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.177186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.491578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.812716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.132223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.450364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.779210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.093262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.435748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.740386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.042413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.347197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.660276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.949721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.243491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.546872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.881287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.196608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.518709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.827899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.148245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.452969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.753991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.051262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.347615] LustreError: 162723:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2851.351227] LustreError: 162723:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2851.351700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.661209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.975060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.293678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.649351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.976194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.288555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.590309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.909491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.214975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.525699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.836141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.161821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.487506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.827813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.158939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.485365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.784317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.098096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.397082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.728236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.015171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.307573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.619141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.948813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.251637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.557079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.875292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.188567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.492113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.791805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.077706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.383135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.691157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.013192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.329747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.652972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.952082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.264936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.586322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.912887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.211547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.535054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.855905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.205976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.524573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.853543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.173466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.504643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.819520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.118912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.419427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.722221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.006679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.300891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.597078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.909224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.216489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.515770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.824207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.161779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.482857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.803342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.105431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.441845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.748510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.043884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.346343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.665759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.964954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.275721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.564876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.870273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.164205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.467830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.775085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.094034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.399587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.704529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.008719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.310488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.598562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.900163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.186187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.492906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.786251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.077602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.367649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.678190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.975774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.285506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.588645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.906769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.204127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.522332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.832959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.165211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.467017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.778129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.091953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.435629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.748130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.064693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.377568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.709089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.035047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.351996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.677296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.996434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.292049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.592882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.898767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.224554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.530677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.857992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.154927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.483390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.804054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.116195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.426273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.769489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.090775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.402639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.712957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.046295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.344664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.655672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.951555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.258676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.560428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.862968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.172712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.477449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.770705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.065352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.353880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.674845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.976778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.290824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.594703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.915930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.227808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.520991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.839462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.172445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.486993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.799572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.105802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.413693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.712675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.994389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.288372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.608525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.914909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.219021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.522394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.844385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.150536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.452479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.753567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.055313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.356473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.667376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.982130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.310799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.622493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.942852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.267655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.580651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.892813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.195485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.480795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.785571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.097597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.411150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.710610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.037731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.336137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.626260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.934768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.267950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.568759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.846895] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2907.848788] Lustre: Skipped 409 previous similar messages [ 2907.853501] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2907.857050] Lustre: Skipped 409 previous similar messages [ 2907.858490] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2907.860269] Lustre: Skipped 409 previous similar messages [ 2907.861654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.864676] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2907.866156] Lustre: Skipped 409 previous similar messages [ 2908.152082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.475470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.801738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.126654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.454369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.780579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.064335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.352222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.660330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.964505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.293450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.687982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.027653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.461476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.867504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.252750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.559935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.854433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.163522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.470248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.769294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.102230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.429307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.784597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.086482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.414980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.743305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.061576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.353892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.660686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.961152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.297122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.612285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.907415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.219809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.541107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.848321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.159705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.478099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.791978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.082737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.413335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.723708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.046118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.350843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.660323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.958995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.293516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.599553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.911989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.218281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.533767] LustreError: 162723:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2924.537082] LustreError: 162723:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2924.538519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.841522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.160872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.489219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.838306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.141297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.435460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.736481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.042277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.338268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.649291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.972466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.292686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.597390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.919655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.231406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.568504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.894986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.219285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.543461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.893358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.210664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.503548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.799487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.122597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.436104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.751767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.130616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.462712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.764912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.080575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.397744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.729801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.049289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.345563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.645045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.959141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.270956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.588307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.891539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.241640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.554535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.874982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.194335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.518475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.823017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.133810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.445304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.779410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.093726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.421491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.756229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.128114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.442298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.776118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.090787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.418544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.722609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.035551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.338150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.655550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.969623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.280362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.607384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.927196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.232185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.545214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.860111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.203622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.495892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.809385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.131613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.471068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.786416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.105441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.433241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.780721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.094499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.403679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.703205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.044890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.364977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.695555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.014330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.361607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.684956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.997504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.322482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.660764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.966095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.277568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.574211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.878356] LustreError: 162723:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 2953.881839] LustreError: 162723:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 2953.882724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.193894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.498256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.797443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.115445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.424068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.741303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.044382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.369206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.665473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.964991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.291256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.626377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.946528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.248183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.572057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.919798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.238857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.570932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.886348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.215788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.526117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.843566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.159427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.488216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.801934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.104403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.417525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.035888] LustreError: 164796:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3016.039129] LustreError: 162723:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3016.042187] LustreError: 162723:0:(osp_precreate.c:651:osp_precreate_send()) Skipped 1 previous similar message [ 3078.623840] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3078.626442] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3080.261063] LustreError: 216442:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3080.336296] Lustre: server umount lustre-OST0000 complete [ 3081.692232] LustreError: 216642:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3081.694063] LustreError: 216642:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3081.784520] Lustre: server umount lustre-MDT0000 complete [ 3082.210131] LustreError: 195768:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3082.214574] LustreError: 195768:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3082.988325] LustreError: 162682:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768011175 with bad export cookie 15648998090288170417 [ 3082.993553] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3092.454068] LustreError: 216842:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3092.457232] LustreError: 216842:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3092.525796] Lustre: server umount lustre-MDT0001 complete [ 3095.967195] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3097.045602] Key type lgssc unregistered [ 3097.179407] LNet: 217375:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3097.183206] LNetError: 217375:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3097.191429] LNet: Removed LNI 192.168.204.143@tcp [ 3097.494551] Key type .llcrypt unregistered [ 3097.495953] Key type ._llcrypt unregistered [ 3106.662104] Key type ._llcrypt registered [ 3106.663426] Key type .llcrypt registered [ 3106.707529] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3107.093909] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3107.102648] alg: No test for adler32 (adler32-zlib) [ 3107.960540] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 3108.049773] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3109.639223] Key type lgssc registered [ 3109.992871] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3112.594483] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3112.600522] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3113.762114] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3115.087972] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3118.055476] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3118.215872] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3119.629689] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3121.454609] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3122.644417] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3124.884194] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3125.010827] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3126.890732] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3128.658381] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3138.343938] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 3141.388210] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3142.017489] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3142.023407] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3142.027120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.030477] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3142.030484] LustreError: 218767:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3143.216080] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3143.217601] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3143.218726] LustreError: Skipped 1 previous similar message [ 3143.226438] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3144.197726] Lustre: lustre-OST0000: Not available for connect from 192.168.204.43@tcp (stopping) [ 3144.200247] Lustre: Skipped 1 previous similar message [ 3145.260089] LustreError: 221167:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3145.299188] Lustre: server umount lustre-OST0000 complete [ 3162.111128] LustreError: 218767:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3173.345995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3173.471845] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3175.438308] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3177.288898] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3183.720361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.724194] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.725428] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3183.731342] Lustre: Skipped 1 previous similar message [ 3185.417495] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 7 sec [ 3186.600563] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8da699102000.ost_server_uuid 50 [ 3187.098411] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8da699102000.ost_server_uuid in FULL state after 0 sec [ 3189.220332] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3189.221075] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3189.224128] Lustre: Skipped 1 previous similar message [ 3189.226220] LustreError: Skipped 5 previous similar messages [ 3189.228278] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3189.230496] Lustre: Skipped 1 previous similar message [ 3194.221928] LustreError: 222493:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3194.225163] LustreError: 222493:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3194.266442] Lustre: server umount lustre-OST0000 complete [ 3199.456555] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3199.457234] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3199.460721] Lustre: Skipped 2 previous similar messages [ 3199.463772] Lustre: Skipped 1 previous similar message [ 3201.664568] LustreError: 222694:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3201.667433] LustreError: 222694:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3201.705270] Lustre: server umount lustre-MDT0000 complete [ 3203.044559] LustreError: 221667:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768011295 with bad export cookie 1481520460761268649 [ 3203.048318] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3203.048492] LustreError: 221667:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3203.197523] Lustre: server umount lustre-MDT0001 complete [ 3206.923737] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3208.081048] Key type lgssc unregistered [ 3208.205470] LNet: 223426:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3208.208324] LNetError: 223426:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3208.216360] LNet: Removed LNI 192.168.204.143@tcp [ 3208.519605] Key type .llcrypt unregistered [ 3208.521339] Key type ._llcrypt unregistered [ 3220.083902] Key type ._llcrypt registered [ 3220.085045] Key type .llcrypt registered [ 3220.124097] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3220.490965] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3220.541072] alg: No test for adler32 (adler32-zlib) [ 3221.415619] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 3221.508852] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3223.095182] Key type lgssc registered [ 3223.494676] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3226.659179] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3226.665543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3227.840561] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3229.150052] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3232.099496] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3232.226710] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3233.520529] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3234.449766] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3236.685780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3236.800449] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3237.863081] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 3239.406176] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3242.229339] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3242.250636] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3242.306198] Lustre: lustre-OST0001: new disk, initializing [ 3242.307904] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3242.337869] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3244.156757] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3246.023227] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3246.028181] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3246.062465] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3249.559431] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3254.198573] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3266.530420] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3266.532233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3266.536103] Lustre: Skipped 3 previous similar messages [ 3266.537964] Lustre: Skipped 2 previous similar messages [ 3271.649314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3271.651398] Lustre: Skipped 2 previous similar messages [ 3272.576389] LustreError: 228469:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3272.612424] Lustre: server umount lustre-MDT0000 complete [ 3273.861221] LustreError: 228191:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768011366 with bad export cookie 1082229848198553313 [ 3273.864994] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3273.866473] LustreError: 228191:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3273.941279] LustreError: 228671:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 3273.945850] LustreError: 228671:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3273.996736] Lustre: server umount lustre-MDT0001 complete [ 3286.005118] LustreError: 228872:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3286.007477] LustreError: 228872:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3286.029541] Lustre: server umount lustre-OST0000 complete [ 3297.589092] LustreError: 229074:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3297.590993] LustreError: 229074:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3297.636164] Lustre: server umount lustre-OST0001 complete [ 3298.702693] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3298.729859] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3299.097822] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3299.129679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3299.494757] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3299.519878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3299.902789] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3299.933972] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3304.628027] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3308.130558] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3308.308366] LustreError: 230481:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3308.316040] LustreError: 230481:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3308.332158] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3309.695311] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3312.569095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3314.028233] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3315.047925] Lustre: 231590:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3317.265043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3317.383484] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3317.385708] Lustre: Skipped 1 previous similar message [ 3318.435278] LustreError: 231948:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3318.439836] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 3319.248390] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3322.171544] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3324.216966] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3327.316545] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3328.548570] Lustre: 233439:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3338.207863] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3338.210227] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3338.214405] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3339.969695] LustreError: 233746:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3339.971848] LustreError: 233746:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3340.011408] Lustre: server umount mylustre-MDT0000 complete [ 3341.308579] LustreError: 231207:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768011434 with bad export cookie 1082229848198556064 [ 3341.312000] LustreError: 231207:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3341.312185] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3343.469722] Lustre: server umount mylustre-MDT0001 complete [ 3354.996073] LustreError: 234147:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 3354.997869] LustreError: 234147:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 3355.014685] Lustre: server umount mylustre-OST0000 complete [ 3366.623468] Lustre: server umount mylustre-OST0001 complete [ 3367.704636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3367.733501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3368.088990] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3368.129095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3368.493676] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3368.514047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3368.880806] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3368.916447] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3373.797424] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3377.257066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3377.410661] LustreError: 235756:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3377.433097] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3377.436152] Lustre: Skipped 1 previous similar message [ 3378.767677] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3381.650412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3383.150324] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3384.124315] Lustre: 236867:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3386.387934] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3387.556231] LustreError: 237222:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3387.559842] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49793) [ 3388.459450] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3391.425711] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3393.458352] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3396.509126] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3397.668287] Lustre: 238711:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3407.327883] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3407.331435] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3407.335549] Lustre: Skipped 3 previous similar messages [ 3407.338242] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3407.340650] Lustre: Skipped 3 previous similar messages [ 3410.691778] LustreError: 238960:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3410.694088] LustreError: 238960:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 3410.728940] Lustre: server umount tfs-MDT0000 complete [ 3411.943137] LustreError: 235732:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768011504 with bad export cookie 1082229848198557730 [ 3411.947163] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3411.947470] LustreError: 235732:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3412.059540] Lustre: server umount tfs-MDT0001 complete [ 3423.555257] Lustre: server umount tfs-OST0000 complete [ 3435.055966] Lustre: server umount tfs-OST0001 complete [ 3436.137323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3436.164155] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3436.521161] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3436.547576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3436.890046] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3436.917740] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3437.268048] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3437.296886] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3442.140884] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3445.767312] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3445.919828] LustreError: 240973:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3445.941651] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3445.944658] Lustre: Skipped 3 previous similar messages [ 3447.177343] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3450.131230] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3451.543620] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3452.535807] Lustre: 242085:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3454.732940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3455.909692] LustreError: 242441:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3455.917560] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49825) [ 3456.813417] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3459.711569] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3461.728209] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3464.877680] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3466.270232] Lustre: 243930:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3473.497759] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 21:19:25 (1768011565) [ 3486.175819] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3486.177958] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3486.181524] Lustre: Skipped 3 previous similar messages [ 3486.183132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3486.185375] Lustre: Skipped 3 previous similar messages [ 3491.392163] LustreError: 244297:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3491.395431] LustreError: 244297:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3491.427183] Lustre: server umount lustre-MDT0000 complete [ 3491.809471] LustreError: 241690:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3491.814992] LustreError: 241690:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3492.589334] LustreError: 240949:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768011585 with bad export cookie 1082229848198559172 [ 3492.594910] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3496.929224] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3496.930117] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3496.932987] Lustre: Skipped 4 previous similar messages [ 3496.935144] Lustre: Skipped 3 previous similar messages [ 3498.809971] Lustre: server umount lustre-MDT0001 complete [ 3514.335101] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3514.368286] Lustre: server umount lustre-OST0000 complete [ 3521.685192] Lustre: server umount lustre-OST0001 complete [ 3523.724359] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 3526.951734] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3530.055836] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3532.114175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3533.980387] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3535.988673] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3538.060683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3538.080464] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3538.080464] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3538.080464] [ 3538.085481] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3538.165713] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3538.176680] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3538.219299] Lustre: lustre-MDT0000: new disk, initializing [ 3538.243678] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3538.245751] Lustre: Skipped 3 previous similar messages [ 3538.251226] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3539.449297] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3543.305175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3543.327537] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3543.358917] Lustre: 247092:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3543.384967] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3543.386613] Lustre: Skipped 1 previous similar message [ 3543.421484] Lustre: lustre-MDT0001: new disk, initializing [ 3543.469834] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3543.473110] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3544.817311] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3548.823631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3548.844618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3548.938990] Lustre: lustre-OST0000: new disk, initializing [ 3548.940954] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3550.138398] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3550.141312] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3550.174636] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3550.661154] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3554.616996] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3554.640145] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3554.701218] Lustre: lustre-OST0001: new disk, initializing [ 3554.703205] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3556.570462] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3556.686306] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3556.689186] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3556.730830] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3560.814455] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 21:20:52 (1768011652) [ 3564.468664] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 21:20:56 (1768011656) [ 3577.320665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3577.324765] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3577.326776] Lustre: Skipped 4 previous similar messages [ 3590.111111] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3590.133173] LustreError: 250512:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3590.135025] LustreError: 250512:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3590.152438] Lustre: server umount lustre-OST0000 complete [ 3592.674747] LustreError: 248629:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3592.681047] LustreError: 248629:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3597.596385] Lustre: server umount lustre-MDT0000 complete [ 3598.762325] LustreError: 247083:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768011691 with bad export cookie 1082229848198560502 [ 3598.765552] LustreError: 247083:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 3598.766047] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3602.912410] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3602.915644] Lustre: Skipped 4 previous similar messages [ 3613.151100] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3613.235126] Lustre: server umount lustre-MDT0001 complete [ 3616.657754] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3634.143125] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3634.201733] Lustre: server umount lustre-OST0001 complete [ 3635.974651] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 3642.017406] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3645.451720] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3647.391228] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3649.148740] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3650.960689] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3652.979039] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3652.999098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3653.076855] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3653.086364] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3653.120805] Lustre: lustre-MDT0000: new disk, initializing [ 3653.147365] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3654.322201] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3658.129725] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3658.151303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3658.181604] Lustre: 253975:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3658.273432] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3659.442605] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3662.350445] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3663.477619] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3665.569691] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3665.601546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3667.358701] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3667.642171] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3667.676888] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3670.210376] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3673.060132] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3673.060695] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3673.062238] LustreError: Skipped 1 previous similar message [ 3673.062921] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3673.065153] Lustre: Skipped 1 previous similar message [ 3673.066216] Lustre: Skipped 10 previous similar messages [ 3678.546680] Lustre: server umount lustre-OST0000 complete [ 3685.855685] Lustre: server umount lustre-MDT0000 complete [ 3687.036455] LustreError: 253967:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768011779 with bad export cookie 1082229848198561972 [ 3687.042200] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3687.161905] Lustre: server umount lustre-MDT0001 complete [ 3690.662617] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3691.626873] Key type lgssc unregistered [ 3691.747336] LNet: 257348:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3691.749468] LNetError: 257348:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3691.756407] LNet: Removed LNI 192.168.204.143@tcp [ 3692.047119] Key type .llcrypt unregistered [ 3692.048924] Key type ._llcrypt unregistered [ 3700.891734] Key type ._llcrypt registered [ 3700.892806] Key type .llcrypt registered [ 3700.926710] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3701.305452] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3701.336592] alg: No test for adler32 (adler32-zlib) [ 3702.174316] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 3702.251449] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3703.823076] Key type lgssc registered [ 3704.129480] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3706.376127] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3706.381708] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3707.525624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3708.643824] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3712.162970] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3712.309593] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3713.964880] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3716.374500] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3717.908164] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3720.810263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3720.945278] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3723.327629] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3725.596208] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3726.090464] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3727.334821] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3727.335904] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3727.344397] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3729.009192] LustreError: 261061:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3729.069159] Lustre: server umount lustre-OST0000 complete [ 3732.449069] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3732.450137] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3732.456695] Lustre: Skipped 1 previous similar message [ 3732.623801] LustreError: 261262:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3732.628456] LustreError: 261262:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3732.660682] Lustre: server umount lustre-MDT0000 complete [ 3737.567967] LustreError: 258769:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3737.571825] LustreError: 258769:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3742.687995] LustreError: 258769:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3742.691937] LustreError: 258769:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3747.808050] LustreError: 258769:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3747.812584] LustreError: 258769:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3749.343122] Lustre: 258760:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1768011826/real 1768011826] req@0000000028131e1b x1853894734925312/t0(0) o101->MGC192.168.204.143@tcp@0@lo:26/25 lens 328/344 e 0 to 1 dl 1768011842 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3749.349339] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3749.352638] LustreError: 258760:0:(mgc_request.c:614:do_requeue()) failed processing log: -5 [ 3749.353069] LustreError: 258078:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000430408ad x1853894734926976/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3749.418952] LustreError: 261463:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3749.421274] LustreError: 261463:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3769.823286] LustreError: 258760:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 [ 3769.827943] LustreError: 261463:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3769.829784] LustreError: 261463:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3769.871881] Lustre: server umount lustre-MDT0001 complete [ 3773.345466] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3774.266808] Key type lgssc unregistered [ 3774.383387] LNet: 262000:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3774.385429] LNetError: 262000:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3774.396389] LNet: Removed LNI 192.168.204.143@tcp [ 3774.688110] Key type .llcrypt unregistered [ 3774.689165] Key type ._llcrypt unregistered [ 3781.212501] Key type ._llcrypt registered [ 3781.213463] Key type .llcrypt registered [ 3781.247226] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3781.555075] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3781.562380] alg: No test for adler32 (adler32-zlib) [ 3782.410349] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 3782.492347] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3784.063117] Key type lgssc registered [ 3784.400444] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3787.246079] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3787.251929] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3802.655255] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3807.775179] LustreError: 263151:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.143@tcp: failed processing log, type 4: rc = -110 [ 3834.399124] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3836.096210] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3837.775188] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3838.456090] LustreError: 263814:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 3838.496971] Lustre: server umount lustre-OST0000 complete [ 3843.644790] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3844.670262] Key type lgssc unregistered [ 3844.789429] LNet: 264650:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3844.791675] LNetError: 264650:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3844.799354] LNet: Removed LNI 192.168.204.143@tcp [ 3845.074756] Key type .llcrypt unregistered [ 3845.076562] Key type ._llcrypt unregistered [ 3857.853730] Key type ._llcrypt registered [ 3857.854723] Key type .llcrypt registered [ 3857.885112] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3858.195776] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3858.239931] alg: No test for adler32 (adler32-zlib) [ 3859.083189] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 3859.195689] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3860.791142] Key type lgssc registered [ 3861.144460] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3864.941562] loop0: detected capacity change from 0 to 204800000 [ 3864.995836] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3864.998777] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3865.107151] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3865.127474] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3865.420808] loop0: detected capacity change from 0 to 204800000 [ 3865.474783] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3865.477895] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3865.588949] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3865.607164] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3865.913515] loop0: detected capacity change from 0 to 204800000 [ 3865.964822] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3865.967781] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3866.005286] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3866.045101] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3866.345254] loop0: detected capacity change from 0 to 204800000 [ 3866.396772] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3866.479039] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3866.774335] loop0: detected capacity change from 0 to 204800000 [ 3866.785021] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3867.994985] loop0: detected capacity change from 0 to 204800000 [ 3868.004085] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3869.223615] loop0: detected capacity change from 0 to 204800000 [ 3869.232191] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3870.437948] loop0: detected capacity change from 0 to 204800000 [ 3870.447888] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3871.660664] loop0: detected capacity change from 0 to 204800000 [ 3871.697493] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3871.718746] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3871.725291] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3871.747704] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3871.750646] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3871.753505] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3871.756653] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3871.760393] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3871.764275] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3871.768413] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3871.772352] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3871.776251] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3871.780275] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3871.784532] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3871.788298] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3871.792221] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3871.796236] Lustre: 268023:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3874.202946] LustreError: 268289:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3874.247705] Lustre: server umount MGS complete [ 3874.519492] loop0: detected capacity change from 0 to 204800000 [ 3874.555042] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3874.707100] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3874.721470] Lustre: lustre-MDT0000: Aborting client recovery [ 3875.010261] loop1: detected capacity change from 0 to 204800000 [ 3875.043080] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3875.070850] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3875.104976] Lustre: 268458:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3875.108538] Lustre: 268458:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3875.111610] Lustre: 268458:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3875.114927] Lustre: 268458:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3875.118161] Lustre: 268458:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3875.121456] Lustre: 268458:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3875.124467] Lustre: 268458:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3875.130470] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3875.223592] Lustre: lustre-MDT0001: Aborting client recovery [ 3875.510627] loop2: detected capacity change from 0 to 204800000 [ 3875.540774] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3875.583535] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3875.598469] Lustre: 268542:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3875.601454] Lustre: 268542:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3875.604638] Lustre: 268542:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3875.607521] Lustre: 268542:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3875.610470] Lustre: 268542:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3875.613523] Lustre: 268542:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3875.616403] Lustre: 268542:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3875.662872] Lustre: Found index 0 for lustre-OST0000, updating log [ 3875.692031] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3875.693814] Lustre: Skipped 1 previous similar message [ 3875.698915] Lustre: lustre-OST0000: Aborting recovery [ 3875.984613] loop3: detected capacity change from 0 to 204800000 [ 3876.037309] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3876.059600] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3876.073996] Lustre: 268629:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3876.077086] Lustre: 268629:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3876.079976] Lustre: 268629:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3876.083159] Lustre: 268629:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3876.086261] Lustre: 268629:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3876.089451] Lustre: 268629:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3876.092542] Lustre: 268629:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3881.447487] LustreError: 268401:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 3884.514374] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3884.514731] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3884.515865] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3884.520311] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3884.524765] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3884.530297] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3886.571601] LustreError: 268367:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 197us: rc = -2 [ 3899.873452] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3899.874298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3899.876643] Lustre: Skipped 2 previous similar messages [ 3899.878233] Lustre: Skipped 1 previous similar message [ 3903.738077] LustreError: 268986:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3903.740426] LustreError: 268986:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 3903.769598] Lustre: server umount lustre-MDT0000 complete [ 3904.046459] LustreError: 268350:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768011996 with bad export cookie 1578337619628298623 [ 3904.047990] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3904.050692] LustreError: 268350:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3904.991567] LustreError: 268396:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3904.991939] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3904.992275] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3904.992282] Lustre: Skipped 2 previous similar messages [ 3904.997192] LustreError: 268396:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3905.000394] Lustre: Skipped 2 previous similar messages [ 3910.111687] LustreError: 268396:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3910.111818] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3910.118099] LustreError: 268396:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3910.119687] Lustre: Skipped 2 previous similar messages [ 3910.202258] LustreError: 269037:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 3910.204023] LustreError: 269037:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3910.258530] Lustre: server umount lustre-MDT0001 complete [ 3916.662100] LustreError: 269089:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 3916.666440] LustreError: 269089:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3916.703120] Lustre: server umount lustre-OST0000 complete [ 3923.183995] LustreError: 269141:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3923.189780] LustreError: 269141:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3923.275928] Lustre: server umount lustre-OST0001 complete [ 3926.913953] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 21:26:58 (1768012018) [ 3938.333849] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 3942.917655] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3947.372573] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3950.189027] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3953.038707] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3956.223896] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3959.594740] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3959.631986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3959.734680] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3959.745653] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3959.788950] Lustre: lustre-MDT0000: new disk, initializing [ 3959.811460] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3959.813253] Lustre: Skipped 1 previous similar message [ 3959.818246] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3961.507425] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3967.020297] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3967.051402] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3967.106595] Lustre: 272688:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3967.148344] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3967.152210] Lustre: Skipped 1 previous similar message [ 3967.203539] Lustre: lustre-MDT0001: new disk, initializing [ 3967.240315] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3967.255804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3967.260122] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3968.789142] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3972.706237] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3974.178403] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3977.315257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3977.345535] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3977.456144] Lustre: lustre-OST0000: new disk, initializing [ 3977.457984] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3977.487511] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3979.778361] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3982.342842] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3982.348103] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3982.391828] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3983.723838] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3986.053700] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3990.429354] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3995.645479] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4001.483174] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4038.630776] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4038.631427] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4038.646186] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4038.655223] Lustre: Skipped 1 previous similar message [ 4041.394188] LustreError: 276156:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4041.398057] LustreError: 276156:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4041.455938] Lustre: server umount lustre-OST0000 complete [ 4043.745503] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4043.746584] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4043.753901] Lustre: Skipped 2 previous similar messages [ 4049.544358] Lustre: server umount lustre-MDT0000 complete [ 4051.424659] LustreError: 272678:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768012144 with bad export cookie 1578337619628301325 [ 4051.434463] LustreError: 272678:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4051.435830] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4051.621178] Lustre: server umount lustre-MDT0001 complete [ 4055.706642] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4057.551856] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4059.776248] LustreError: 277502:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4059.778010] LustreError: 277502:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 4059.807123] Lustre: server umount MGS complete [ 4065.276526] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4065.479321] LustreError: 278124:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4065.486795] LustreError: 278136:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4065.517842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4067.237440] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4070.881865] LustreError: 278124:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4071.524947] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4073.267619] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4075.891925] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4077.752906] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4080.705456] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4083.168865] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4085.588635] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4116.962349] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4116.964981] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4116.966796] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4116.966804] Lustre: Skipped 3 previous similar messages [ 4116.967622] LustreError: Skipped 1 previous similar message [ 4116.983361] Lustre: Skipped 1 previous similar message [ 4122.990097] LustreError: 280805:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4122.992212] LustreError: 280805:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4123.024138] Lustre: server umount lustre-OST0000 complete [ 4127.200802] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4127.204469] Lustre: Skipped 1 previous similar message [ 4130.396857] Lustre: server umount lustre-MDT0000 complete [ 4131.559126] LustreError: 278103:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768012224 with bad export cookie 1578337619628303152 [ 4131.561472] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4131.561972] LustreError: 278103:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4131.672358] Lustre: server umount lustre-MDT0001 complete [ 4135.342527] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 4136.303146] Key type lgssc unregistered [ 4136.437401] LNet: 281738:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4136.440347] LNetError: 281738:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4136.452398] LNet: Removed LNI 192.168.204.143@tcp [ 4136.765844] Key type .llcrypt unregistered [ 4136.767233] Key type ._llcrypt unregistered [ 4148.904145] Key type ._llcrypt registered [ 4148.905071] Key type .llcrypt registered [ 4148.936977] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 4155.168472] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 4155.507936] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4155.539265] alg: No test for adler32 (adler32-zlib) [ 4156.392778] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 4156.484195] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 4158.063145] Key type lgssc registered [ 4158.480352] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4161.729353] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4164.092499] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4166.405531] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4168.916668] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4171.715222] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4171.728553] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4171.732709] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4172.815191] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4172.828168] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4172.865374] Lustre: lustre-MDT0000: new disk, initializing [ 4172.890320] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4172.896794] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4174.298348] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4178.517385] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4178.557608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4178.616939] Lustre: 285643:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4178.659553] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4178.663112] Lustre: Skipped 1 previous similar message [ 4178.716254] Lustre: lustre-MDT0001: new disk, initializing [ 4178.763485] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4178.780437] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4178.785755] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4180.338434] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4183.649073] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4185.124001] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4187.471543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4187.493473] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4187.591900] Lustre: lustre-OST0000: new disk, initializing [ 4187.593887] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4187.625218] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4189.835236] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4189.888706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4189.893075] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4189.938603] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4192.934875] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4196.475684] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4202.985539] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4207.786286] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4246.499086] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4246.504195] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4246.508592] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4246.512438] Lustre: Skipped 1 previous similar message [ 4251.623362] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4251.627702] Lustre: Skipped 2 previous similar messages [ 4256.223088] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4256.239130] LustreError: 289204:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4256.285210] Lustre: server umount lustre-OST0000 complete [ 4260.832615] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4260.839034] LustreError: Skipped 1 previous similar message [ 4260.842454] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4260.852982] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4261.858737] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4264.003080] LustreError: 289406:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4264.006543] LustreError: 289406:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4264.035081] Lustre: server umount lustre-MDT0000 complete [ 4265.355978] LustreError: 286423:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768012358 with bad export cookie 16602100758484963426 [ 4265.361115] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4265.446493] LustreError: 289608:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4265.448908] LustreError: 289608:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4265.518952] Lustre: server umount lustre-MDT0001 complete [ 4269.156898] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4270.797862] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4272.879665] LustreError: 290549:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4272.884223] LustreError: 290549:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4272.917646] Lustre: server umount MGS complete [ 4277.774354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4278.021489] LustreError: 291184:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4278.053397] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4279.569404] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4283.341658] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4283.361981] LustreError: 291172:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4283.374145] LustreError: 291172:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4283.539540] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4285.299850] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4287.750416] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4289.457941] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4292.523945] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4292.710491] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4294.923895] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4296.612421] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4328.930584] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4328.932505] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4328.934148] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4328.937119] Lustre: Skipped 1 previous similar message [ 4328.940915] Lustre: Skipped 3 previous similar messages [ 4334.002141] LustreError: 293852:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4334.006474] LustreError: 293852:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4334.069234] Lustre: server umount lustre-OST0000 complete [ 4339.170161] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4339.171191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4339.178411] Lustre: Skipped 1 previous similar message [ 4339.185652] Lustre: Skipped 1 previous similar message [ 4342.002773] Lustre: server umount lustre-MDT0000 complete [ 4343.681179] LustreError: 291197:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768012436 with bad export cookie 16602100758484965239 [ 4343.688414] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4343.689817] LustreError: 291197:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 4343.791612] LustreError: 294256:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4343.795184] LustreError: 294256:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4343.857199] Lustre: server umount lustre-MDT0001 complete [ 4348.586301] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 4349.906651] Key type lgssc unregistered [ 4350.026587] LNet: 294787:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4350.031252] LNetError: 294787:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4350.041595] LNet: Removed LNI 192.168.204.143@tcp [ 4350.372390] Key type .llcrypt unregistered [ 4350.373700] Key type ._llcrypt unregistered [ 4358.652706] Key type ._llcrypt registered [ 4358.654106] Key type .llcrypt registered [ 4358.707826] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 4366.099095] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 4366.598294] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4366.611790] alg: No test for adler32 (adler32-zlib) [ 4367.466705] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 4367.565753] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 4369.151248] Key type lgssc registered [ 4369.679290] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4373.700934] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4376.771069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4380.319407] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 4383.357712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4386.013474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4386.043741] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4386.052607] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4387.150951] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4387.165187] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4387.212709] Lustre: lustre-MDT0000: new disk, initializing [ 4387.239536] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4387.246614] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4388.708187] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4393.326102] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4393.362818] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4393.490286] Lustre: lustre-OST0000: new disk, initializing [ 4393.492343] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4393.494965] Lustre: Skipped 1 previous similar message [ 4393.519867] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4394.263173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4394.267769] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4394.278564] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4395.535722] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4573.665144] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4573.672895] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4577.030533] LustreError: 299697:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 4577.131486] Lustre: server umount lustre-MDT0000 complete [ 4578.770351] LustreError: 297950:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768012671 with bad export cookie 11339336828116857389 [ 4578.774327] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4578.777225] LustreError: 297950:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4578.817834] LustreError: 299900:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4578.821410] LustreError: 299900:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4578.884136] Lustre: server umount lustre-OST0000 complete [ 4588.489503] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 4592.122074] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 4595.715669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4598.356628] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4600.905198] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4603.454505] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4606.061355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4606.090497] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4606.210417] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4606.226868] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4606.273754] Lustre: lustre-MDT0000: new disk, initializing [ 4606.298470] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4606.305784] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4607.724237] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4612.497834] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4612.537268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4612.599827] Lustre: 302940:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4612.643619] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4612.647211] Lustre: Skipped 1 previous similar message [ 4612.694899] Lustre: lustre-MDT0001: new disk, initializing [ 4612.733604] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4612.752941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4612.759277] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4614.381494] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4617.956635] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4619.330658] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4622.234468] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4622.269172] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4622.394876] Lustre: lustre-OST0000: new disk, initializing [ 4622.397577] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4622.436502] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4624.626287] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4628.085854] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4629.822307] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4631.565965] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4631.572411] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4631.628890] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4632.016833] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 4633.679566] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4633.767297] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4636.641847] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4636.645182] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4636.649158] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4636.650518] Lustre: Skipped 1 previous similar message [ 4640.688184] LustreError: 305894:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4640.693884] LustreError: 305894:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4640.761389] Lustre: server umount lustre-OST0000 complete [ 4643.808470] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4643.812996] LustreError: Skipped 1 previous similar message [ 4643.815649] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4643.823354] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4643.826988] Lustre: Skipped 1 previous similar message [ 4646.881769] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4646.882819] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4648.588295] LustreError: 306095:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4648.591715] LustreError: 306095:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4648.630245] Lustre: server umount lustre-MDT0000 complete [ 4650.198579] LustreError: 302930:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768012742 with bad export cookie 11339336828119798166 [ 4650.203568] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4650.204907] LustreError: 302930:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4650.316899] Lustre: server umount lustre-MDT0001 complete [ 4651.550251] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 4652.129555] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 21:39:03 (1768012743) [ 4654.723443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4654.984382] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4656.611635] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4659.781685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4661.421898] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4663.169396] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4664.357926] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4666.478784] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4668.462987] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4670.438482] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4672.937358] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4672.971427] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4673.051003] Lustre: lustre-OST0001: new disk, initializing [ 4673.054099] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4673.125182] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4673.128527] Lustre: Skipped 2 previous similar messages [ 4673.178915] LustreError: 309399:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4674.212367] LustreError: 309399:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4675.233275] LustreError: 309399:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4675.566774] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4677.283166] LustreError: 309399:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4677.289613] LustreError: 309399:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4678.637078] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4678.641933] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4679.817334] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4679.879891] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4681.072342] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8da685b6d000.ost_server_uuid 50 [ 4681.376981] LustreError: 309399:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4681.379671] LustreError: 309399:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [ 4681.520174] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8da685b6d000.ost_server_uuid in FULL state after 0 sec [ 4682.560930] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8da685b6d000.ost_server_uuid 50 [ 4682.993563] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8da685b6d000.ost_server_uuid in FULL state after 0 sec [ 4683.428438] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4695.012510] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4695.013521] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4695.015516] Lustre: Skipped 1 previous similar message [ 4695.021081] Lustre: Skipped 2 previous similar messages [ 4700.782105] LustreError: 310685:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 4700.784961] LustreError: 310685:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 4700.803587] Lustre: server umount lustre-OST0001 complete [ 4705.249882] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4705.251948] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4705.254856] Lustre: Skipped 1 previous similar message [ 4705.255488] LustreError: 308466:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4705.255541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4705.255547] Lustre: Skipped 3 previous similar messages [ 4705.267671] LustreError: 308466:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4708.467694] Lustre: server umount lustre-OST0000 complete [ 4716.312240] LustreError: 311089:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 4716.316760] LustreError: 311089:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4716.367795] Lustre: server umount lustre-MDT0000 complete [ 4718.125877] LustreError: 306849:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768012810 with bad export cookie 11339336828119799034 [ 4718.132116] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4718.284732] Lustre: server umount lustre-MDT0001 complete [ 4721.873356] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 4723.144298] Key type lgssc unregistered [ 4723.288468] LNet: 311821:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4723.290530] LNetError: 311821:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4723.301369] LNet: Removed LNI 192.168.204.143@tcp [ 4723.619992] Key type .llcrypt unregistered [ 4723.621643] Key type ._llcrypt unregistered [ 4732.476720] Key type ._llcrypt registered [ 4732.477646] Key type .llcrypt registered [ 4732.510677] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 4732.803417] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4732.809307] alg: No test for adler32 (adler32-zlib) [ 4733.672876] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 4733.775401] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 4735.359234] Key type lgssc registered [ 4735.941563] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4739.233807] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4739.245477] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4740.417198] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4742.064717] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4744.048696] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4747.101221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4747.328849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4748.776685] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4753.413979] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4753.594866] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4756.082782] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4757.670300] LustreError: 314752:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4757.681405] LustreError: 314752:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4758.667845] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4761.904252] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4762.029520] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4764.172212] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4766.131202] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4766.140499] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4766.501208] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 4769.513162] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4769.591205] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4848.097144] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4848.102825] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4848.107586] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4853.219800] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4853.221313] Lustre: Skipped 2 previous similar messages [ 4858.335120] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4858.338637] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4858.342484] Lustre: Skipped 1 previous similar message [ 4858.357072] LustreError: 316620:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 4858.372957] Lustre: server umount lustre-OST0001 complete [ 4863.457396] LustreError: 315250:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4863.458059] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4863.460339] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4863.460345] Lustre: Skipped 1 previous similar message [ 4863.460994] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4863.463407] LustreError: 315250:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4868.579093] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4868.579202] LustreError: 314753:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4868.580914] Lustre: Skipped 2 previous similar messages [ 4868.585359] LustreError: 314753:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4873.702460] LustreError: 316146:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4873.712142] LustreError: 316146:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4874.207187] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4874.232137] LustreError: 316822:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4874.236363] LustreError: 316822:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4874.295619] Lustre: server umount lustre-OST0000 complete [ 4878.817307] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4878.818377] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4878.822131] Lustre: Skipped 2 previous similar messages [ 4878.826401] Lustre: Skipped 3 previous similar messages [ 4882.241795] LustreError: 317026:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4882.245366] LustreError: 317026:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4882.277075] Lustre: server umount lustre-MDT0000 complete [ 4883.586704] LustreError: 314774:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768012976 with bad export cookie 2222085843763362747 [ 4883.592083] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4883.593460] LustreError: 314774:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4883.744211] Lustre: server umount lustre-MDT0001 complete [ 4888.310287] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 4889.306581] Key type lgssc unregistered [ 4889.461608] LNet: 317761:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4889.466305] LNetError: 317761:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4889.476700] LNet: Removed LNI 192.168.204.143@tcp [ 4889.956299] Key type .llcrypt unregistered [ 4889.958560] Key type ._llcrypt unregistered [ 4911.207569] Key type ._llcrypt registered [ 4911.208701] Key type .llcrypt registered [ 4911.260009] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 4911.774679] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4911.785294] alg: No test for adler32 (adler32-zlib) [ 4912.652799] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 4912.743357] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 4914.343179] Key type lgssc registered [ 4914.910686] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4919.478254] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4919.492085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4920.701698] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4922.122688] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4925.470225] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4925.617464] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4927.052583] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4928.308261] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4931.668192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4931.812073] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4934.219764] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4935.910430] LustreError: 321469:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4935.917035] LustreError: 321469:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4938.748064] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4938.879474] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4941.522157] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4945.643734] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 4945.645220] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 4950.858104] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4960.321077] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4971.492666] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4971.495768] Lustre: Skipped 2 previous similar messages [ 4971.496803] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4971.498530] Lustre: Skipped 2 previous similar messages [ 4975.166660] LustreError: 323420:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4975.194233] Lustre: server umount lustre-MDT0000 complete [ 4976.377869] LustreError: 321116:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768013069 with bad export cookie 7278319941413443343 [ 4976.382386] LustreError: 321116:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4976.383876] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4976.449751] LustreError: 323621:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 4976.451670] LustreError: 323621:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4976.506452] Lustre: server umount lustre-MDT0001 complete [ 4987.954087] LustreError: 323822:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4987.956161] LustreError: 323822:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4987.973765] Lustre: server umount lustre-OST0000 complete [ 4998.321908] LustreError: 324023:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4998.323519] LustreError: 324023:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4998.366438] Lustre: server umount lustre-OST0001 complete [ 5003.868414] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5006.974334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5007.106530] LustreError: 325200:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5007.123898] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5008.277504] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5010.940635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5012.303100] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5013.178810] Lustre: 326307:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5015.173278] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5015.304851] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5015.306827] Lustre: Skipped 1 previous similar message [ 5016.354736] LustreError: 326663:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5016.358465] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5017.085741] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5019.715382] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5020.846776] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5021.473514] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5024.483436] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5025.671466] Lustre: 328155:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5031.903794] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5031.906059] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5031.909316] Lustre: Skipped 1 previous similar message [ 5031.910817] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5036.514197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5036.515200] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5036.519056] Lustre: Skipped 2 previous similar messages [ 5037.695654] LustreError: 328259:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5037.697444] LustreError: 328259:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5037.727551] Lustre: server umount lustre-MDT0000 complete [ 5038.855265] LustreError: 325179:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768013131 with bad export cookie 7278319941413445030 [ 5038.858329] LustreError: 325179:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5038.859128] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5038.962788] Lustre: server umount lustre-MDT0001 complete [ 5049.854143] LustreError: 328661:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5049.859421] LustreError: 328661:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5049.887717] Lustre: server umount lustre-OST0000 complete [ 5062.014201] Lustre: server umount lustre-OST0001 complete [ 5063.629685] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5064.232266] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5064.870143] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 21:45:56 (1768013156) [ 5073.181497] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5076.662706] loop0: detected capacity change from 0 to 1099511627776 [ 5076.829696] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5076.832340] *** VALIDATE xfs *** [ 5076.836133] XFS (loop0): Mounting V5 Filesystem [ 5076.844376] XFS (loop0): Ending clean mount [ 5077.468331] loop1: detected capacity change from 0 to 18691697672192 [ 5091.355036] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5098.807330] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 5103.087457] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5106.695786] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5109.128268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5111.613167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5114.730693] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5118.137980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5118.186615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5118.320924] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5118.338784] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5118.387796] Lustre: lustre-MDT0000: new disk, initializing [ 5118.424762] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5118.428014] Lustre: Skipped 1 previous similar message [ 5118.437475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5120.061316] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5125.430482] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5125.466056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5125.526994] Lustre: 333231:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5125.571671] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5125.575609] Lustre: Skipped 1 previous similar message [ 5125.623516] Lustre: lustre-MDT0001: new disk, initializing [ 5125.671491] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5125.674596] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5127.280065] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5131.393448] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5133.021633] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5136.160258] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5136.196756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5136.333622] Lustre: lustre-OST0000: new disk, initializing [ 5136.336591] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5137.422906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5137.429912] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5137.482729] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5138.807125] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5142.468060] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5143.883390] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5143.986331] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5145.285827] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5145.347870] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5147.620218] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5147.621902] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5147.623429] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5147.623437] Lustre: Skipped 3 previous similar messages [ 5147.629174] Lustre: Skipped 1 previous similar message [ 5147.642368] LustreError: Skipped 1 previous similar message [ 5152.050881] LustreError: 336169:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5152.055398] LustreError: 336169:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5152.113288] Lustre: server umount lustre-OST0000 complete [ 5156.320593] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5156.324633] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5156.331160] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5156.334530] Lustre: Skipped 1 previous similar message [ 5159.602984] Lustre: server umount lustre-MDT0000 complete [ 5161.378452] LustreError: 333222:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768013254 with bad export cookie 7278319941413446584 [ 5161.382280] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5161.382793] LustreError: 333222:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5161.517535] Lustre: server umount lustre-MDT0001 complete [ 5165.035880] XFS (loop0): Unmounting Filesystem [ 5166.928073] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 21:47:38 (1768013258) [ 5170.069323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5170.251058] LustreError: 337390:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5170.282440] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5170.285851] Lustre: Skipped 2 previous similar messages [ 5171.955903] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5175.778041] LustreError: 337390:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5176.118798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5178.051574] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5180.435002] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5182.078292] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5185.444523] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5185.509610] LustreError: 338988:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5188.310276] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5191.085097] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5202.912535] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5202.917569] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5202.925258] Lustre: Skipped 1 previous similar message [ 5202.929112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5202.933039] Lustre: Skipped 2 previous similar messages [ 5206.510094] LustreError: 339733:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5206.512343] LustreError: 339733:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 5206.550111] Lustre: server umount lustre-OST0000 complete [ 5214.447147] Lustre: server umount lustre-MDT0000 complete [ 5215.200906] LustreError: 337390:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5215.212271] LustreError: 337390:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5216.345832] LustreError: 337371:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768013309 with bad export cookie 7278319941413447466 [ 5216.351678] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5216.352460] LustreError: 337371:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5216.479403] Lustre: server umount lustre-MDT0001 complete [ 5221.680023] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 5223.168083] Key type lgssc unregistered [ 5223.336689] LNet: 340669:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5223.340999] LNetError: 340669:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5223.352692] LNet: Removed LNI 192.168.204.143@tcp [ 5223.863180] Key type .llcrypt unregistered [ 5223.865521] Key type ._llcrypt unregistered [ 5236.283130] Key type ._llcrypt registered [ 5236.285486] Key type .llcrypt registered [ 5236.333542] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5236.739674] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5236.745789] alg: No test for adler32 (adler32-zlib) [ 5237.610489] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g8820be3 [ 5237.697588] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 5239.279204] Key type lgssc registered [ 5239.843985] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5243.175651] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5243.186547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5244.371125] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5246.032422] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5249.345139] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5249.465954] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5250.875450] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5252.878321] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5254.268857] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5256.741186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5256.843382] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5258.856063] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5261.425790] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5275.619929] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5275.628859] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5277.201728] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.43@tcp (stopping) [ 5280.224682] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5280.230893] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5280.238946] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5281.390198] LustreError: 344508:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5281.420798] Lustre: server umount lustre-MDT0001 complete [ 5282.304630] LustreError: 342190:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5282.317922] LustreError: 342190:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5282.955117] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5285.858873] LustreError: 342190:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5285.864667] LustreError: 342190:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5286.344310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5286.391447] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5286.395163] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5286.413478] Lustre: 342182:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5286.541812] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5288.199905] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5292.016812] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5296.111284] LustreError: 345655:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5296.117346] LustreError: 345655:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5296.123701] LustreError: 342185:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5296.158299] LustreError: 345661:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.204.143@tcp: cfg command failed: rc = -17 [ 5296.166451] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5296.166451] [ 5594.080553] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5594.085673] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5594.094540] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5594.098536] Lustre: Skipped 1 previous similar message [ 5597.154327] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5599.741818] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.43@tcp (stopping) [ 5599.746446] Lustre: Skipped 3 previous similar messages [ 5606.367211] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5606.430112] LustreError: 356279:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5606.433581] LustreError: 356279:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5606.459726] Lustre: server umount lustre-MDT0001 complete [ 5607.394918] LustreError: 344350:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5607.403129] LustreError: 344350:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5607.591734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5609.981694] LustreError: 344248:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5610.894954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5610.936405] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5610.938386] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5610.951119] Lustre: 342182:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5611.077202] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5612.725298] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5616.107469] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5621.241483] LustreError: 357465:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5621.247678] LustreError: 357465:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5621.253835] LustreError: 342185:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5621.259192] LustreError: 342185:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5621.300426] LustreError: 357473:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 5621.306034] LustreError: 357473:0:(obd_config.c:797:class_detach()) OBD device 8 still set up [ 5621.310370] LustreError: 357473:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.204.143@tcp: cfg command failed: rc = -17 [ 5621.315183] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.143@tcp [ 5621.315183] [ 5917.195222] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.43@tcp (stopping) [ 5917.201749] Lustre: Skipped 3 previous similar messages [ 5918.183303] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5918.184200] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5922.002786] LustreError: 367848:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5922.006955] LustreError: 367848:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5922.050987] Lustre: server umount lustre-MDT0001 complete [ 5922.302298] LustreError: 342189:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5923.448336] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5927.229887] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5927.288669] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5927.291846] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5927.318756] Lustre: 342182:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5927.507989] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 5929.393240] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5932.526946] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5937.132143] LustreError: 369025:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5937.138106] LustreError: 369025:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5937.144467] LustreError: 342185:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5937.149158] LustreError: 342185:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5937.181442] LustreError: 369028:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 5937.190541] LustreError: 369028:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.204.143@tcp: cfg command failed: rc = -114 [ 5937.196132] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 5937.196132] [ 6249.958614] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6249.962099] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6249.962736] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6249.962746] Lustre: Skipped 3 previous similar messages [ 6249.964127] Lustre: Skipped 2 previous similar messages [ 6251.157683] LustreError: 379591:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6251.160615] LustreError: 379591:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6251.199462] Lustre: server umount lustre-MDT0000 complete [ 6252.868461] LustreError: 345076:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768014345 with bad export cookie 16283309091353552956 [ 6252.878854] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6255.072754] LustreError: 344248:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6255.080309] LustreError: 344248:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6267.359134] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6267.417367] LustreError: 379794:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6267.420853] LustreError: 379794:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6267.481812] Lustre: server umount lustre-MDT0001 complete [ 6268.956874] Lustre: server umount lustre-OST0000 complete [ 6272.682659] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 6277.219982] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6281.846990] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6285.003212] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6287.839289] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6291.180729] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6293.998559] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 22:06:25 (1768014385) [ 6297.228425] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6297.261988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6297.374054] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6297.389143] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6297.437953] Lustre: lustre-MDT0000: new disk, initializing [ 6297.472124] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6297.483443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6299.428661] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6304.978064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6305.022384] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6305.076910] Lustre: 382529:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6305.121475] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6305.124790] Lustre: Skipped 1 previous similar message [ 6305.179213] Lustre: lustre-MDT0001: new disk, initializing [ 6305.224515] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6305.244964] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6305.250622] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6307.234363] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6311.133316] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6312.882210] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6316.277547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6316.320094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6316.484766] Lustre: lustre-OST0000: new disk, initializing [ 6316.488411] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6316.536806] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6319.308835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6319.312956] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6319.362315] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6319.366477] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6323.177952] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6329.828375] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6329.834635] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6329.835920] Lustre: Skipped 4 previous similar messages [ 6329.839854] Lustre: Skipped 7 previous similar messages [ 6332.626207] LustreError: 384971:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6332.630212] LustreError: 384971:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 6332.668318] Lustre: server umount lustre-MDT0000 complete [ 6334.387912] LustreError: 382519:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1768014427 with bad export cookie 16283309091353554909 [ 6334.395068] LustreError: 382519:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 6334.397552] LustreError: 383322:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6334.397865] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6334.591198] Lustre: server umount lustre-MDT0001 complete [ 6346.727921] Lustre: server umount lustre-OST0000 complete [ 6358.737837] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 6363.140528] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6367.327231] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6370.252498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6372.874839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6376.253982] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6377.178297] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6189 sec ========== 22:07:48 (1768014468) [ 6378.070155] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 22:07:49 (1768014469) === [ 6379.703389] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 22:07:51 (1768014471) === [ 6398.251565] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 6399.731803] Key type lgssc unregistered [ 6399.906613] LNet: 390475:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6399.911649] LNetError: 390475:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6399.923571] LNet: Removed LNI 192.168.204.143@tcp [ 6400.385551] Key type .llcrypt unregistered [ 6400.387881] Key type ._llcrypt unregistered