[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 403839284 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002372] x2apic enabled [ 0.003005] Switched APIC routing to physical x2apic. [ 0.004000] kvm-guest: setup PV IPIs [ 0.005361] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.006017] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.007008] pid_max: default: 32768 minimum: 301 [ 0.008130] LSM: Security Framework initializing [ 0.009042] Yama: becoming mindful. [ 0.010027] SELinux: Initializing. [ 0.011071] *** VALIDATE selinux *** [ 0.018709] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.023414] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.024150] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.025095] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026096] *** VALIDATE tmpfs *** [ 0.028177] *** VALIDATE proc *** [ 0.029194] *** VALIDATE cgroup *** [ 0.030004] *** VALIDATE cgroup2 *** [ 0.031206] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.032112] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.033003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.034020] Spectre V2 : User space: Vulnerable [ 0.035004] Speculative Store Bypass: Vulnerable [ 0.038155] debug: unmapping init [mem 0xffffffff8dc59000-0xffffffff8dc60fff] [ 0.041000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.041591] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.042015] ... version: 2 [ 0.043000] ... bit width: 48 [ 0.044006] ... generic registers: 4 [ 0.045007] ... value mask: 0000ffffffffffff [ 0.046007] ... max period: 00007fffffffffff [ 0.047007] ... fixed-purpose events: 3 [ 0.048007] ... event mask: 000000070000000f [ 0.049273] rcu: Hierarchical SRCU implementation. [ 0.051464] smp: Bringing up secondary CPUs ... [ 0.052493] x86: Booting SMP configuration: [ 0.053013] .... node #0, CPUs: #1 #2 #3 [ 0.056320] smp: Brought up 1 node, 4 CPUs [ 0.058007] smpboot: Max logical packages: 1 [ 0.059010] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.086047] node 0 deferred pages initialised in 26ms [ 0.089369] devtmpfs: initialized [ 0.090204] x86/mm: Memory block size: 128MB [ 0.093084] gcov: version magic: 0x41383552 [ 0.095115] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.096070] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.097212] pinctrl core: initialized pinctrl subsystem [ 0.098147] [ 0.098736] ************************************************************* [ 0.099008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.100010] ** ** [ 0.101009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.102008] ** ** [ 0.103008] ** This means that this kernel is built to expose internal ** [ 0.104007] ** IOMMU data structures, which may compromise security on ** [ 0.105009] ** your system. ** [ 0.106007] ** ** [ 0.107008] ** If you see this message and you are not debugging the ** [ 0.108009] ** kernel, report this immediately to your vendor! ** [ 0.109013] ** ** [ 0.110007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.111006] ************************************************************* [ 0.112771] NET: Registered protocol family 16 [ 0.113506] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.114036] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.115036] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.116585] cpuidle: using governor menu [ 0.118918] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.121618] PCI: Using configuration type 1 for base access [ 0.124137] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.135133] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.136010] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.137164] cryptd: max_cpu_qlen set to 1000 [ 0.140315] ACPI: Added _OSI(Module Device) [ 0.141012] ACPI: Added _OSI(Processor Device) [ 0.142007] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.143010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.147718] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.151344] ACPI: Interpreter enabled [ 0.152156] ACPI: PM: (supports S0 S3 S4 S5) [ 0.153009] ACPI: Using IOAPIC for interrupt routing [ 0.154114] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.155394] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.166791] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.167018] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.168009] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.169058] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.171490] acpiphp: Slot [2] registered [ 0.172075] acpiphp: Slot [5] registered [ 0.173066] acpiphp: Slot [6] registered [ 0.174074] acpiphp: Slot [7] registered [ 0.175083] acpiphp: Slot [8] registered [ 0.176145] acpiphp: Slot [9] registered [ 0.177078] acpiphp: Slot [10] registered [ 0.178067] acpiphp: Slot [3] registered [ 0.179076] acpiphp: Slot [4] registered [ 0.180066] acpiphp: Slot [11] registered [ 0.181060] acpiphp: Slot [12] registered [ 0.182063] acpiphp: Slot [13] registered [ 0.183070] acpiphp: Slot [14] registered [ 0.185085] acpiphp: Slot [15] registered [ 0.186068] acpiphp: Slot [16] registered [ 0.187146] acpiphp: Slot [17] registered [ 0.189069] acpiphp: Slot [18] registered [ 0.190173] acpiphp: Slot [19] registered [ 0.192059] acpiphp: Slot [20] registered [ 0.193075] acpiphp: Slot [21] registered [ 0.194000] acpiphp: Slot [22] registered [ 0.194000] acpiphp: Slot [23] registered [ 0.195145] acpiphp: Slot [24] registered [ 0.197063] acpiphp: Slot [25] registered [ 0.198157] acpiphp: Slot [26] registered [ 0.200062] acpiphp: Slot [27] registered [ 0.201142] acpiphp: Slot [28] registered [ 0.203077] acpiphp: Slot [29] registered [ 0.205075] acpiphp: Slot [30] registered [ 0.209105] acpiphp: Slot [31] registered [ 0.210079] PCI host bridge to bus 0000:00 [ 0.212096] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.214012] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.216012] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.218010] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.220010] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.223014] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.224147] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.228034] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.231315] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.239848] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.244531] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.247023] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.249010] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.252014] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.256297] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.262735] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.267096] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.273312] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x1e0 took 10742 usecs [ 0.278733] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.290012] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.309013] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.317000] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.324000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.333012] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.338013] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.363023] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.384993] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.392012] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.398014] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.413012] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.421853] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.427012] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.432009] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.441014] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.445991] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.449012] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.452009] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.459000] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.465279] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.474030] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.481014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.502022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.511882] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.520014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.525013] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.539017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.552441] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.555303] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.557287] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.560348] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.563223] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.567117] iommu: Default domain type: Passthrough [ 0.569717] SCSI subsystem initialized [ 0.571181] ACPI: bus type USB registered [ 0.573081] usbcore: registered new interface driver usbfs [ 0.576076] usbcore: registered new interface driver hub [ 0.579094] usbcore: registered new device driver usb [ 0.581158] pps_core: LinuxPPS API ver. 1 registered [ 0.583009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.587140] PTP clock support registered [ 0.590032] EDAC MC: Ver: 3.0.0 [ 0.593118] PCI: Using ACPI for IRQ routing [ 0.595764] NetLabel: Initializing [ 0.597008] NetLabel: domain hash size = 128 [ 0.599007] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.601082] NetLabel: unlabeled traffic allowed by default [ 0.604337] vgaarb: loaded [ 0.605225] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.607009] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.615594] clocksource: Switched to clocksource kvm-clock [ 0.746706] VFS: Disk quotas dquot_6.6.0 [ 0.748613] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.751508] *** VALIDATE ramfs *** [ 0.752916] *** VALIDATE hugetlbfs *** [ 0.754618] pnp: PnP ACPI init [ 0.757715] pnp: PnP ACPI: found 6 devices [ 0.773393] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.777497] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.780025] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.782565] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.785371] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.788057] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.791078] NET: Registered protocol family 2 [ 0.794021] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.800634] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.804785] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.810822] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.814904] TCP: Hash tables configured (established 65536 bind 65536) [ 0.818447] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.822609] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.826081] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.829651] NET: Registered protocol family 1 [ 0.832854] RPC: Registered named UNIX socket transport module. [ 0.835516] RPC: Registered udp transport module. [ 0.837707] RPC: Registered tcp transport module. [ 0.840765] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.844321] NET: Registered protocol family 44 [ 0.846512] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.849226] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.851819] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.854918] PCI: CLS 0 bytes, default 64 [ 0.856838] Unpacking initramfs... [ 2.534284] debug: unmapping init [mem 0xffff9e1ffcc54000-0xffff9e1ffffbffff] [ 2.539335] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.542073] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.545947] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.096277] Initialise system trusted keyrings [ 3.099513] Key type blacklist registered [ 3.101499] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.114841] zbud: loaded [ 3.118452] *** VALIDATE nfs *** [ 3.119786] *** VALIDATE nfs4 *** [ 3.124072] pstore: using deflate compression [ 3.129692] Platform Keyring initialized [ 3.272310] NET: Registered protocol family 38 [ 3.274959] Key type asymmetric registered [ 3.278760] Asymmetric key parser 'x509' registered [ 3.283938] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.288472] io scheduler mq-deadline registered [ 3.291953] io scheduler kyber registered [ 3.294243] io scheduler bfq registered [ 3.296773] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.300206] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.304300] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.308376] ACPI: Power Button [PWRF] [ 3.315946] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.323342] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.338505] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.348611] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.384701] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.413269] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.442810] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.450073] Non-volatile memory driver v1.3 [ 3.453498] Linux agpgart interface v0.103 [ 3.499561] virtio_blk virtio1: [vda] 134032 512-byte logical blocks (68.6 MB/65.4 MiB) [ 3.503246] vda: detected capacity change from 0 to 68624384 [ 3.537679] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.541822] vdb: detected capacity change from 0 to 1073741824 [ 3.572319] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.579880] vdc: detected capacity change from 0 to 2621440000 [ 3.609432] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.613505] vdd: detected capacity change from 0 to 2621440000 [ 3.635044] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.639502] vde: detected capacity change from 0 to 4294967296 [ 3.668869] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.674960] vdf: detected capacity change from 0 to 4294967296 [ 3.691984] libphy: Fixed MDIO Bus: probed [ 3.704862] usbcore: registered new interface driver usbserial_generic [ 3.708058] usbserial: USB Serial support registered for generic [ 3.710672] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.715947] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.717895] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.720724] mousedev: PS/2 mouse device common for all mice [ 3.724231] rtc_cmos 00:05: RTC can wake from S4 [ 3.727199] rtc_cmos 00:05: registered as rtc0 [ 3.728942] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.730128] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.739991] intel_pstate: CPU model not supported [ 3.742861] hid: raw HID events driver (C) Jiri Kosina [ 3.750318] usbcore: registered new interface driver usbhid [ 3.752731] usbhid: USB HID core driver [ 3.755243] drop_monitor: Initializing network drop monitor service [ 3.760574] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.761415] Initializing XFRM netlink socket [ 3.772542] NET: Registered protocol family 10 [ 3.775447] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.780420] Segment Routing with IPv6 [ 3.782786] NET: Registered protocol family 17 [ 3.784886] mpls_gso: MPLS GSO support [ 3.790898] RAS: Correctable Errors collector initialized. [ 3.793297] AVX version of gcm_enc/dec engaged. [ 3.795436] AES CTR mode by8 optimization enabled [ 3.882663] sched_clock: Marking stable (3882508891, 0)->(4751007824, -868498933) [ 3.889436] registered taskstats version 1 [ 3.892254] Loading compiled-in X.509 certificates [ 3.894690] zswap: loaded using pool lzo/zbud [ 3.921777] Key type big_key registered [ 3.940880] Key type encrypted registered [ 3.943393] ima: No TPM chip found, activating TPM-bypass! [ 3.946956] ima: Allocated hash algorithm: sha1 [ 3.949775] ima: No architecture policies found [ 3.952875] evm: Initialising EVM extended attributes: [ 3.955983] evm: security.selinux [ 3.957956] evm: security.ima [ 3.959808] evm: security.capability [ 3.961856] evm: HMAC attrs: 0x1 [ 3.965397] rtc_cmos 00:05: setting system clock to 2026-01-02 00:45:57 UTC (1767314757) [ 3.974870] debug: unmapping init [mem 0xffffffff8ec03000-0xffffffff8edfffff] [ 3.979558] debug: unmapping init [mem 0xffffffff8d982000-0xffffffff8dc58fff] [ 3.990130] Write protecting the kernel read-only data: 28672k [ 3.995632] debug: unmapping init [mem 0xffffffff8c003000-0xffffffff8c1fffff] [ 3.999758] debug: unmapping init [mem 0xffffffff8c914000-0xffffffff8c9fffff] [ 4.035619] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.047226] systemd[1]: Detected virtualization kvm. [ 4.049551] systemd[1]: Detected architecture x86-64. [ 4.051917] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.080963] systemd[1]: No hostname configured. [ 4.084504] systemd[1]: Set hostname to . [ 4.086911] random: systemd: uninitialized urandom read (16 bytes read) [ 4.089940] systemd[1]: Initializing machine ID from random generator. [ 4.158572] random: ln: uninitialized urandom read (6 bytes read) [ 4.282507] random: systemd: uninitialized urandom read (16 bytes read) [ 4.285912] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.297880] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 4.306355] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. [ OK ] Reached target Timers. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.175810] device-mapper: uevent: version 1.0.3 [ 5.177746] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [[ 5.617307] random: fast init done  OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 6.682811] virtio_net virtio0 ens2: renamed from eth0 [ 6.982360] scsi host0: ata_piix [ 7.014861] scsi host1: ata_piix [ 7.021471] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.029266] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 11.826841] dracut-initqueue[585]: RTNETLINK answers: File exists [ 12.472943] random: crng init done [ 12.475498] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 13.207723] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 14.681029] printk: systemd: 26 output lines suppressed due to ratelimiting [ 15.093981] SELinux: Disabled at runtime. [ 15.188747] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 15.203890] systemd[1]: Detected virtualization kvm. [ 15.207890] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 16.035664] systemd[1]: initrd-switch-root.service: Succeeded. [ 16.038817] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 16.053213] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 16.058290] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 16.061108] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 16.069316] systemd[1]: Starting Journal Service... Starting Journal Service... [ 16.082174] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice User and Session Slice. Mounting Huge Pages File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Mounting POSIX Message Queue File System... [ OK ] Reached target RPC Port Mapper. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. Activating swap /dev/disk/by-label/SWAP... Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ 16.281665] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 16.791849] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 17.306634] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 17.325641] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 17.577808] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 17.608969] EDAC sbridge: Ver: 1.1.2 [ 19.884138] Key type dns_resolver registered [ 20.407693] NFS: Registering the id_resolver key type [ 20.411267] Key type id_resolver registered [ 20.413423] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Restore /run/initramfs on shutdown... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target sshd-keygen.target. [ OK ] Started dnf makecache --timer. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg307-server login: [ 49.275221] libcfs: loading out-of-tree module taints kernel. [ 49.309628] Key type ._llcrypt registered [ 49.311366] Key type .llcrypt registered [ 49.395354] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 93.345374] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 93.936913] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 93.943468] alg: No test for adler32 (adler32-zlib) [ 94.978501] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 95.412556] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 97.056134] Key type lgssc registered [ 97.975229] Lustre: Echo OBD driver; http://www.lustre.org/ [ 113.112917] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 113.435039] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 113.700872] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 114.232177] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 114.754448] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 146.983904] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 163.181207] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 163.191309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 163.226647] Lustre: lustre-MDT0001: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 163.229225] Lustre: Skipped 1 previous similar message [ 163.236385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 163.241166] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 166.375121] Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0001 [ 166.402142] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 166.464209] Lustre: lustre-MDT0001: new disk, initializing [ 166.509747] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 166.518439] Lustre: srv-lustre-MDT0001: Waiting to contact MDT0000 to allocate super-sequence: rc = -115 [ 166.521981] Lustre: 15506:0:(lod_sub_object.c:951:lod_sub_prep_llog()) lustre-MDT0001-mdtlov: can't create new llog: rc = -115 [ 166.525642] LustreError: 15506:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osd: get update log duration 0, retries 0, failed: rc = -115 [ 166.541659] Lustre: 15486:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0000/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 166.686056] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 167.718080] Lustre: 15551:0:(client.c:1637:after_reply()) @@@ resending request on EINPROGRESS req@000000007d7ca2eb x1853163934064000/t0(0) o700->lustre-MDT0001-osp-MDT0000@0@lo:30/10 lens 264/248 e 0 to 0 dl 1767314937 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'' uid:0 gid:0 projid:4294967295 [ 168.724622] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 168.726263] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 173.065792] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 173.202258] Lustre: 16696:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 176.806962] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 176.812711] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 181.423929] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 181.472530] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 181.511418] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 181.596333] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 181.607621] Lustre: lustre-OST0000: new disk, initializing [ 181.609956] Lustre: Skipped 1 previous similar message [ 181.612983] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 181.617383] Lustre: Skipped 2 previous similar messages [ 181.641537] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 181.644727] Lustre: Skipped 1 previous similar message [ 182.423358] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 182.426875] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 182.452934] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000400 [ 182.485622] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 184.536915] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 184.687990] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 191.362932] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 195.446045] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 203.236967] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing check_logdir /tmp/testlogs/ [ 207.409795] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing yml_node [ 208.600013] hrtimer: interrupt took 7650645 ns [ 217.155878] Lustre: DEBUG MARKER: Client: 2.17.0.RC4 [ 219.279166] Lustre: DEBUG MARKER: MDS: 2.17.0.RC4 [ 223.057091] Lustre: DEBUG MARKER: OSS: 2.17.0.RC4 [ 224.678210] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jan 1 19:49:36 EST 2026 [ 251.494499] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 254.244069] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 280.033964] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 280.042733] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 280.046153] Lustre: Skipped 2 previous similar messages [ 281.411493] LustreError: 22220:0:(obd_class.h:479:obd_check_dev()) Device 26 not setup [ 281.520788] Lustre: server umount lustre-MDT0000 complete [ 281.963845] Lustre: server umount lustre-MDT0001 complete [ 289.620867] LustreError: 19800:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767315043 with bad export cookie 16800118458304928251 [ 289.625904] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 289.627075] LustreError: 19800:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 289.668170] LustreError: 23201:0:(obd_class.h:479:obd_check_dev()) Device 20 not setup [ 289.670029] LustreError: 23201:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 289.709101] Lustre: server umount lustre-OST0000 complete [ 289.846463] Lustre: server umount lustre-OST0001 complete [ 308.954508] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 378.961177] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 400.147585] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 400.859676] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 401.953506] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 404.196332] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 422.180371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 422.277183] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 422.494762] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 422.515091] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 422.518494] Lustre: Skipped 1 previous similar message [ 422.599172] Lustre: lustre-MDT0000: new disk, initializing [ 422.601377] Lustre: Skipped 1 previous similar message [ 422.648553] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 422.650829] Lustre: Skipped 1 previous similar message [ 422.668788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 422.673081] Lustre: Skipped 1 previous similar message [ 426.890479] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 447.067341] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 447.130601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 447.234304] Lustre: 31117:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 447.306972] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 447.313991] Lustre: Skipped 1 previous similar message [ 447.392206] Lustre: lustre-MDT0001: new disk, initializing [ 447.527989] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 447.586300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 447.604473] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 447.612995] Lustre: Skipped 1 previous similar message [ 452.184498] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 459.771870] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 463.254052] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 479.959309] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 480.029443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 480.377504] Lustre: lustre-OST0000: new disk, initializing [ 480.381873] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 480.479685] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 482.089361] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 482.105553] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 482.202337] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 486.266365] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 494.888651] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 499.918595] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 500.185403] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 503.869252] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 504.165126] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 507.875213] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 507.877796] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 507.878178] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 507.899403] Lustre: Skipped 4 previous similar messages [ 507.931233] Lustre: Skipped 2 previous similar messages [ 512.140886] LustreError: 35090:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 512.157935] LustreError: 35090:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 512.476789] Lustre: server umount lustre-OST0000 complete [ 519.545872] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 519.569201] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 524.285681] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 524.291050] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 524.333391] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 533.475336] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 533.478261] Lustre: Skipped 4 previous similar messages [ 533.984350] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 534.097093] LustreError: 35340:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 534.102477] LustreError: 35340:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 534.248239] Lustre: server umount lustre-MDT0000 complete [ 540.420441] LustreError: 33848:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767315293 with bad export cookie 16800118458304929679 [ 540.424445] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 540.427463] LustreError: 33848:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 540.572149] LustreError: 35590:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 540.575320] LustreError: 35590:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 540.705788] Lustre: server umount lustre-MDT0001 complete [ 546.007538] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 19:54:57 (1767315297) [ 556.630240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 557.180381] LustreError: 36310:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 557.204124] LustreError: 36310:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 557.282641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 562.281924] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 562.660795] LustreError: 36310:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 566.755778] LustreError: 36310:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 571.874027] LustreError: 36310:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 576.062152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 576.541510] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 581.809726] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 587.548351] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 591.090205] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 604.059059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 611.932264] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 617.931614] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 632.393663] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 643.555765] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 643.560448] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 643.576042] Lustre: Skipped 1 previous similar message [ 643.594366] Lustre: Skipped 1 previous similar message [ 645.199893] LustreError: 39195:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 645.210537] LustreError: 39195:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 645.289681] Lustre: server umount lustre-MDT0000 complete [ 646.222903] Lustre: server umount lustre-MDT0001 complete [ 652.442331] LustreError: 38924:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767315405 with bad export cookie 16800118458304930561 [ 652.456678] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 652.462574] LustreError: 38924:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 653.036413] Lustre: server umount lustre-OST0000 complete [ 742.741749] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 763.590714] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 763.716213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 764.257692] LustreError: 44999:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 764.357285] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 764.366329] Lustre: Skipped 1 previous similar message [ 769.762537] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 769.881794] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 776.649458] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 776.685508] Lustre: 46088:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 789.874961] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 796.858954] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 796.976686] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 797.174979] Lustre: lustre-OST0001: new disk, initializing [ 797.185918] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 798.169569] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 798.723053] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 798.748711] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 798.936366] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 804.392635] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 815.868600] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 819.886292] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 840.166795] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 840.170532] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 840.188760] Lustre: Skipped 3 previous similar messages [ 840.212604] Lustre: Skipped 3 previous similar messages [ 845.217437] LustreError: 49355:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 845.232607] LustreError: 49355:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 845.391039] Lustre: server umount lustre-MDT0001 complete [ 846.113630] Lustre: server umount lustre-MDT0000 complete [ 850.794242] LustreError: 44982:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767315604 with bad export cookie 16800118458304931422 [ 850.808146] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 850.808379] LustreError: 44982:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 850.991882] Lustre: server umount lustre-OST0001 complete [ 851.341748] Lustre: server umount lustre-OST0000 complete [ 952.489102] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 973.845165] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 974.312646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 974.454310] LustreError: 55182:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 974.591187] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 974.596890] Lustre: Skipped 3 previous similar messages [ 980.343986] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 981.219359] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 988.821320] Lustre: 56265:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1003.943240] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1007.194452] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1012.874812] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1016.875344] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1027.110072] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1031.348615] Lustre: 58738:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1031.366280] Lustre: 58738:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 1045.994099] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1046.011200] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1046.021995] Lustre: Skipped 1 previous similar message [ 1046.037920] Lustre: Skipped 5 previous similar messages [ 1046.223442] LustreError: 59107:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 1046.232531] LustreError: 59107:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 1046.554532] Lustre: server umount lustre-MDT0001 complete [ 1050.411183] Lustre: server umount lustre-MDT0000 complete [ 1054.886170] LustreError: 56268:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767315808 with bad export cookie 16800118458304932696 [ 1054.897448] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1055.006546] Lustre: server umount lustre-OST0000 complete [ 1055.325725] Lustre: server umount lustre-OST0001 complete [ 1067.705717] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 20:03:39 (1767315819) [ 1162.236986] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 1186.937937] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1187.136446] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1187.691120] LustreError: 66055:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1187.859724] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1187.868401] Lustre: Skipped 3 previous similar messages [ 1194.163524] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1194.474920] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1201.667673] Lustre: 67152:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1215.689849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1219.659395] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1224.353394] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1227.433542] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1241.991876] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1245.856480] Lustre: 69599:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1245.865109] Lustre: 69599:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 1258.480090] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1258.498664] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1258.501899] Lustre: Skipped 3 previous similar messages [ 1258.521152] Lustre: Skipped 5 previous similar messages [ 1263.153145] LustreError: 69910:0:(obd_class.h:479:obd_check_dev()) Device 11 not setup [ 1263.166295] LustreError: 69910:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 1263.312217] Lustre: server umount lustre-MDT0000 complete [ 1263.591609] LustreError: 66847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1264.288355] Lustre: server umount lustre-MDT0001 complete [ 1268.723723] LustreError: 66034:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767316022 with bad export cookie 16800118458304933795 [ 1268.738207] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1268.742235] LustreError: 66034:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1268.863957] Lustre: server umount lustre-OST0001 complete [ 1268.919908] Lustre: server umount lustre-OST0000 complete [ 1282.043517] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 20:07:13 (1767316033) [ 1374.794194] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 1397.568763] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1397.575573] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1398.041441] LustreError: 75919:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1398.056359] LustreError: 75919:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1403.749846] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1404.316918] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1410.346250] Lustre: 76991:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1423.683619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1427.781184] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1431.624468] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1436.182157] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1450.328912] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1460.697198] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1460.710659] Lustre: Skipped 1 previous similar message [ 1465.815857] Lustre: lustre-MDT0000: Not available for connect from 192.168.203.7@tcp (stopping) [ 1465.827225] Lustre: Skipped 2 previous similar messages [ 1467.691168] LustreError: 79727:0:(obd_class.h:479:obd_check_dev()) Device 11 not setup [ 1467.696122] LustreError: 79727:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 1468.015211] Lustre: server umount lustre-MDT0000 complete [ 1468.901815] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1468.911915] Lustre: Skipped 1 previous similar message [ 1473.291270] LustreError: 75898:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767316226 with bad export cookie 16800118458304934901 [ 1473.293810] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1473.306280] LustreError: 75898:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1479.726848] Lustre: server umount lustre-MDT0001 complete [ 1493.142405] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1497.568753] LustreError: 7027:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9e20476dea00 x1853163934442624/t0(0) o250->MGC192.168.203.107@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1498.125608] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1498.138095] Lustre: Skipped 7 previous similar messages [ 1502.471219] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1508.354077] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1515.687855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1521.063875] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1521.652736] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1521.660264] Lustre: Skipped 1 previous similar message [ 1527.505049] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1529.471715] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1533.589889] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1535.646452] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1547.240338] LustreError: 80629:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1547.253655] LustreError: 80629:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 23 previous similar messages [ 1547.539705] Lustre: server umount lustre-MDT0000 complete [ 1547.747849] Lustre: server umount lustre-MDT0001 complete [ 1552.983564] LustreError: 75898:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767316306 with bad export cookie 16800118458304936042 [ 1552.984421] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1552.991921] LustreError: 75898:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1553.098236] Lustre: server umount lustre-OST0000 complete [ 1553.415543] Lustre: server umount lustre-OST0001 complete [ 1565.635728] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 20:11:56 (1767316316) [ 1645.582877] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 1666.347071] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1666.365266] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1672.342693] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1672.507636] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1679.234522] Lustre: 89598:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1679.242870] Lustre: 89598:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 2 previous similar messages [ 1691.921530] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1695.155725] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1700.482576] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1703.076743] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1712.226264] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1722.468786] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1736.675325] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1736.679626] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1736.691320] Lustre: Skipped 10 previous similar messages [ 1736.703809] Lustre: Skipped 8 previous similar messages [ 1742.166463] LustreError: 92436:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 1742.169219] LustreError: 92436:0:(obd_class.h:479:obd_check_dev()) Skipped 41 previous similar messages [ 1742.406550] Lustre: server umount lustre-MDT0001 complete [ 1743.191706] Lustre: server umount lustre-MDT0000 complete [ 1748.126486] LustreError: 88484:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767316501 with bad export cookie 16800118458304936868 [ 1748.138728] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1748.748867] Lustre: server umount lustre-OST0000 complete [ 1748.949621] Lustre: server umount lustre-OST0001 complete [ 1763.206795] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 20:15:14 (1767316514) [ 1767.487720] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1769.784411] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 20:15:20 (1767316520) [ 1798.715737] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 1845.069193] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 1864.292835] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1870.199487] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1889.252808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1889.337212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1889.641839] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1889.667685] Lustre: Skipped 1 previous similar message [ 1889.725254] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1889.878524] Lustre: lustre-MDT0000: new disk, initializing [ 1890.040816] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1895.731792] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1904.932165] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1919.990743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1920.110219] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1920.513883] Lustre: lustre-OST0000: new disk, initializing [ 1920.518527] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1920.530504] Lustre: Skipped 1 previous similar message [ 1922.480130] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1922.485727] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1922.562993] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1927.118687] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 1934.893670] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1937.197501] Lustre: DEBUG MARKER: create test files [ 1973.730293] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1977.088743] Lustre: server umount lustre-OST0000 complete [ 1984.105687] Lustre: server umount lustre-MDT0000 complete [ 1999.564838] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 2003.253330] Key type lgssc unregistered [ 2003.668321] LNet: 102382:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2003.673213] LNetError: 102382:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2003.704787] LNet: Removed LNI 192.168.203.107@tcp [ 2004.857790] Key type .llcrypt unregistered [ 2004.860485] Key type ._llcrypt unregistered [ 2078.551970] Key type ._llcrypt registered [ 2078.553919] Key type .llcrypt registered [ 2078.647704] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2079.749790] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2079.944218] alg: No test for adler32 (adler32-zlib) [ 2081.046620] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 2081.270320] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 2082.960265] Key type lgssc registered [ 2084.548657] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2092.863435] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2092.886346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2094.467270] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2099.213977] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2105.572820] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2116.616515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2116.983053] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2119.051919] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 2119.213252] Lustre: 107183:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 2119.233684] Lustre: 107183:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 2119.242305] Lustre: 107183:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2119.248982] Lustre: 107183:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 2119.254941] Lustre: 107183:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2119.262406] Lustre: 107183:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2124.322473] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2130.117612] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2135.187767] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 2140.454184] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 2149.858109] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2149.867104] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2149.879759] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2154.981149] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2160.098424] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2162.144138] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2162.190282] LustreError: 107928:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 2162.307392] Lustre: server umount lustre-OST0000 complete [ 2167.078402] LustreError: 108179:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 2167.090804] LustreError: 108179:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2167.286653] Lustre: server umount lustre-MDT0000 complete [ 2180.166461] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 2182.613686] Key type lgssc unregistered [ 2182.922990] LNet: 108856:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2182.929325] LNetError: 108856:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2182.949444] LNet: Removed LNI 192.168.203.107@tcp [ 2183.703319] Key type .llcrypt unregistered [ 2183.705565] Key type ._llcrypt unregistered [ 2254.530351] Key type ._llcrypt registered [ 2254.532130] Key type .llcrypt registered [ 2254.622460] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2256.041587] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2256.134429] alg: No test for adler32 (adler32-zlib) [ 2257.308643] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 2257.579742] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 2259.337537] Key type lgssc registered [ 2260.831314] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2270.475232] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2270.492761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2272.242564] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2277.443834] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2284.055722] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2295.712807] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2296.280960] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2298.349674] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2298.460907] Lustre: 113660:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 2298.468720] Lustre: 113660:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 2298.477537] Lustre: 113660:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2298.485740] Lustre: 113660:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 2298.498755] Lustre: 113660:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2298.504682] Lustre: 113660:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2302.691977] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2308.202661] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2315.126627] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2329.063658] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2329.080084] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2329.094383] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2334.179343] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2335.174582] LustreError: 114373:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 2335.288244] Lustre: server umount lustre-OST0000 complete [ 2340.953865] LustreError: 114623:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 2340.963543] LustreError: 114623:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2341.281784] Lustre: server umount lustre-MDT0000 complete [ 2355.096380] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 2358.938625] Key type lgssc unregistered [ 2359.360862] LNet: 115301:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2359.365233] LNetError: 115301:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2359.391792] LNet: Removed LNI 192.168.203.107@tcp [ 2360.668272] Key type .llcrypt unregistered [ 2360.670493] Key type ._llcrypt unregistered [ 2399.081149] Key type ._llcrypt registered [ 2399.086442] Key type .llcrypt registered [ 2399.186570] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 2479.101169] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2480.628291] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2480.651441] alg: No test for adler32 (adler32-zlib) [ 2481.774738] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 2482.150741] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 2483.936311] Key type lgssc registered [ 2485.244491] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2508.155399] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2508.579491] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2509.961307] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2512.823110] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2531.005656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2531.073545] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2531.098762] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2532.454658] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2532.550971] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2532.661636] Lustre: lustre-MDT0000: new disk, initializing [ 2532.795460] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2532.858809] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2537.242259] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2555.116822] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2555.189913] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2555.250826] Lustre: 124671:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2555.300982] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2555.305257] Lustre: Skipped 1 previous similar message [ 2555.409481] Lustre: lustre-MDT0001: new disk, initializing [ 2555.518208] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2555.542403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2555.551519] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2560.235266] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2568.443412] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2572.870764] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2586.991621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2587.062174] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2587.336943] Lustre: lustre-OST0000: new disk, initializing [ 2587.343564] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2587.492798] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2593.886833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2593.897704] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2593.988202] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2594.144558] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2602.422949] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2607.802611] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2608.101701] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2613.137034] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2613.397838] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2619.876982] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2619.880935] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2619.882830] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2619.903604] Lustre: Skipped 1 previous similar message [ 2619.912422] LustreError: Skipped 1 previous similar message [ 2621.504415] LustreError: 128303:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2621.940499] Lustre: server umount lustre-OST0000 complete [ 2630.125096] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2630.149281] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2630.149780] Lustre: Skipped 1 previous similar message [ 2630.169542] Lustre: Skipped 2 previous similar messages [ 2633.869275] LustreError: 128553:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2633.882203] LustreError: 128553:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2634.069905] Lustre: server umount lustre-MDT0000 complete [ 2638.968390] LustreError: 124663:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767317392 with bad export cookie 2105318964584103456 [ 2639.003084] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2639.105543] LustreError: 128802:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2639.121290] LustreError: 128802:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2639.306258] Lustre: server umount lustre-MDT0001 complete [ 2647.873308] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 20:29:59 (1767317399) [ 2728.978810] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2750.148473] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2793.473519] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 2860.235404] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 2881.636515] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2882.283296] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2883.661710] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2886.187170] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2901.515739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2901.576842] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2901.750823] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2901.801433] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2901.867481] Lustre: lustre-MDT0000: new disk, initializing [ 2901.908459] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2901.935656] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2905.835383] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2921.595296] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2921.631841] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2921.686858] Lustre: 142575:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2921.727485] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2921.730987] Lustre: Skipped 1 previous similar message [ 2921.795476] Lustre: lustre-MDT0001: new disk, initializing [ 2921.881226] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2921.930559] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2921.946899] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2926.393647] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2934.125568] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2938.286920] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2952.429713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2952.516289] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2952.777510] Lustre: lustre-OST0000: new disk, initializing [ 2952.782923] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2952.876596] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2959.102753] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 2962.463679] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2962.471162] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2962.582956] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2966.132966] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2970.293837] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2970.493871] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2975.585724] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2975.844854] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2978.281755] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2978.290589] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2978.315714] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2982.894199] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2982.895155] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2982.910678] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2982.923826] Lustre: Skipped 1 previous similar message [ 2988.005755] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2988.014316] Lustre: Skipped 1 previous similar message [ 2992.096861] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2992.172649] LustreError: 146208:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2992.181018] LustreError: 146208:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2992.323670] Lustre: server umount lustre-OST0000 complete [ 2998.244178] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2998.252104] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2998.260171] Lustre: Skipped 1 previous similar message [ 2998.275972] Lustre: Skipped 1 previous similar message [ 3003.587334] LustreError: 146458:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3003.595375] LustreError: 146458:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3003.710586] Lustre: server umount lustre-MDT0000 complete [ 3008.482366] LustreError: 142584:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3008.515707] LustreError: 142584:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3008.733659] LustreError: 142568:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767317762 with bad export cookie 2105318964584104338 [ 3008.742563] LustreError: 142568:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3008.746627] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3009.144224] Lustre: server umount lustre-MDT0001 complete [ 3018.697484] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 20:36:09 (1767317769) [ 3031.161619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3031.662204] LustreError: 147576:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3031.758437] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3036.377669] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3037.159554] LustreError: 147576:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3042.291240] LustreError: 147576:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3047.393049] LustreError: 147576:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3049.467466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3055.674983] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3061.987876] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3066.370880] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3077.181621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3077.406363] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3077.412194] Lustre: Skipped 1 previous similar message [ 3084.220286] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3090.279109] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3093.993138] Lustre: 150301:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 4b5da81d-fb73-4df6-8c5f-0577101059a3 at adminstrative request [ 3094.755472] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3094.777102] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 3105.070300] Lustre: MGS: Client 4b5da81d-fb73-4df6-8c5f-0577101059a3 (at 0@lo) reconnecting [ 3105.082178] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x1d3798719eedc4fd to 0x1d3798719eedc79d [ 3105.095445] Lustre: MGC192.168.203.107@tcp: Connection restored to 0@lo (at 0@lo) [ 3140.883900] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3140.959980] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3141.123641] Lustre: lustre-OST0001: new disk, initializing [ 3141.127797] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3147.839622] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3147.933857] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3147.951055] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3148.069394] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3157.288325] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 3163.621557] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3163.634880] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3163.636454] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3163.657788] Lustre: Skipped 3 previous similar messages [ 3174.880415] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3174.941218] LustreError: 151976:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 3174.943973] LustreError: 151976:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3174.980898] Lustre: server umount lustre-OST0001 complete [ 3178.981734] LustreError: 149560:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3178.988487] LustreError: 149560:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3182.050640] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3182.058412] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3182.068170] Lustre: Skipped 1 previous similar message [ 3185.600553] Lustre: server umount lustre-OST0000 complete [ 3193.316972] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3193.319755] LustreError: Skipped 1 previous similar message [ 3199.457820] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3199.469560] Lustre: Skipped 10 previous similar messages [ 3205.092606] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3205.209339] LustreError: 152432:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3205.213915] LustreError: 152432:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3205.303688] Lustre: server umount lustre-MDT0000 complete [ 3208.674334] LustreError: 147576:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3208.710033] LustreError: 147576:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3210.136399] LustreError: 151185:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767317963 with bad export cookie 2105318964584105885 [ 3210.147309] LustreError: 151185:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3210.148141] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3210.526256] Lustre: server umount lustre-MDT0001 complete [ 3220.006180] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 20:39:31 (1767317971) [ 3221.821255] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 3223.852614] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 20:39:35 (1767317975) [ 3225.879671] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 3227.818648] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 20:39:39 (1767317979) [ 3229.994667] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 3231.816607] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 20:39:43 (1767317983) [ 3233.356152] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 3235.216314] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 20:39:46 (1767317986) [ 3238.069374] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 3243.935324] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3245.022693] loop: module loaded [ 3245.044324] loop0: detected capacity change from 0 to 4294967296 [ 3245.089052] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3246.678532] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 3250.451224] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 3254.709186] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 3264.090615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3269.407460] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3275.024109] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3291.068373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3291.184666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3291.226968] LustreError: 155785:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 3291.237395] LustreError: 155785:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 3291.244620] LustreError: 155785:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 3291.258181] LustreError: 155785:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 3326.382446] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 3394.935291] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 3418.128351] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3418.145092] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3419.708801] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3422.323971] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3440.360408] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3440.435382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3440.663488] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3440.694970] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3440.840847] Lustre: lustre-MDT0000: new disk, initializing [ 3440.986186] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3440.988971] Lustre: Skipped 1 previous similar message [ 3441.021526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3446.172099] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3464.939167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3465.026839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3465.173549] Lustre: 164740:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3465.453184] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3472.192788] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3481.873636] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3486.782755] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3503.553108] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3503.628986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3510.063573] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3512.379233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3512.395251] Lustre: Skipped 1 previous similar message [ 3512.406759] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3512.606853] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3518.724896] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3524.069238] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3524.263347] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3529.042543] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3529.294575] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3533.289134] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3533.292048] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3533.319036] Lustre: Skipped 3 previous similar messages [ 3533.333870] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3533.352478] Lustre: Skipped 3 previous similar messages [ 3537.545658] LustreError: 168374:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3537.557263] LustreError: 168374:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 3537.827177] Lustre: server umount lustre-OST0000 complete [ 3557.344162] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3557.900287] Lustre: server umount lustre-MDT0000 complete [ 3558.403517] LustreError: 167189:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3558.426771] LustreError: 167189:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3564.415414] LustreError: 164733:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767318317 with bad export cookie 2105318964584106487 [ 3564.427789] LustreError: 164733:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3564.429322] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3565.170257] Lustre: server umount lustre-MDT0001 complete [ 3575.911299] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 20:45:27 (1767318327) [ 3588.385098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3588.993673] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3588.998897] Lustre: Skipped 2 previous similar messages [ 3593.452615] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3607.318505] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3612.974591] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3619.119241] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3623.429308] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3635.161543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3642.872600] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3649.240629] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3654.374220] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3660.074679] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 5 sec [ 3665.654787] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3665.876995] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3683.851595] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3683.951723] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3684.095666] Lustre: lustre-OST0001: new disk, initializing [ 3684.100803] Lustre: Skipped 2 previous similar messages [ 3684.110799] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3684.118386] Lustre: Skipped 3 previous similar messages [ 3690.710256] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3693.126975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3693.144925] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3693.240362] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3699.693525] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 3705.623666] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3705.837735] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3711.124485] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3711.403758] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3731.981797] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3760.082756] Lustre: Failing over lustre-MDT0000 [ 3760.101531] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3760.106625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3760.128046] Lustre: Skipped 6 previous similar messages [ 3760.136575] Lustre: Skipped 10 previous similar messages [ 3760.391562] LustreError: 175665:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3760.393875] LustreError: 175665:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 3760.476136] Lustre: server umount lustre-MDT0000 complete [ 3765.177067] LustreError: 169786:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.7@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3765.201465] LustreError: 169786:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 3781.600111] Lustre: 121139:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318518/real 1767318518] req@00000000fc7719bd x1853166437300096/t0(0) o400->MGC192.168.203.107@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1767318534 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3781.638448] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3798.256436] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3798.259270] LDISKFS-fs (dm-0): recovery complete [ 3798.277629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3808.825355] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3811.229205] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3811.232706] Lustre: 169786:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3813.870583] Lustre: 169791:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3813.884258] Lustre: 169791:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 1 previous similar message [ 3813.914106] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3814.102885] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3814.450895] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3814.821382] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3814.824936] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 3815.167084] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3815.187267] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 3815.200102] Lustre: 177041:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3815.223615] Lustre: 177041:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 7 previous similar messages [ 3815.350408] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 3815.879172] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3815.888948] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 3816.214275] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3816.230849] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 3817.242254] Lustre: 177041:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3817.257821] Lustre: 177041:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 5 previous similar messages [ 3817.913642] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3817.938436] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 5 previous similar messages [ 3818.260381] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3818.267988] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 5 previous similar messages [ 3821.458167] Lustre: 177041:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3821.470614] Lustre: 177041:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 3822.119551] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3822.130820] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 3822.464171] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3822.472746] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 3829.474938] Lustre: 177041:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3829.487679] Lustre: 177041:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 24 previous similar messages [ 3830.129267] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3830.153612] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 24 previous similar messages [ 3830.480691] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3830.489781] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 24 previous similar messages [ 3845.524510] Lustre: 177041:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3845.536570] Lustre: 177041:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 49 previous similar messages [ 3846.202728] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3846.217025] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 49 previous similar messages [ 3846.553283] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3846.568181] LustreError: 177041:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 49 previous similar messages [ 3871.088313] Lustre: 177041:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3871.095254] Lustre: 177041:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3871.101152] Lustre: 177041:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 26b1d94e-a0bd-42b8-b29e-83bf317a9085@192.168.203.7@tcp [ 3871.108537] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3871.113318] Lustre: 177041:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@000000001e84bf41 x1853166353191296/t0(8589934774) o35->26b1d94e-a0bd-42b8-b29e-83bf317a9085@192.168.203.7@tcp:290/0 lens 392/0 e 0 to 0 dl 1767318635 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3871.139646] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3871.171153] Lustre: lustre-MDT0000: Denying connection for new client 26b1d94e-a0bd-42b8-b29e-83bf317a9085 (at 192.168.203.7@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3871.219083] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3871.267501] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3871.268044] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3871.293165] Lustre: Skipped 2 previous similar messages [ 3871.366756] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3871.368205] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 3887.588742] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3887.595652] LustreError: Skipped 1 previous similar message [ 3888.615589] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3888.636879] Lustre: Skipped 1 previous similar message [ 3899.872187] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3899.934207] LustreError: 177701:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3899.936796] LustreError: 177701:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3899.999962] Lustre: server umount lustre-OST0000 complete [ 3900.917621] LustreError: 171779:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3900.961060] LustreError: 171779:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 63 previous similar messages [ 3911.894358] Lustre: server umount lustre-OST0001 complete [ 3922.873272] Lustre: server umount lustre-MDT0000 complete [ 3927.162763] LustreError: 169773:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767318680 with bad export cookie 2105318964584157706 [ 3927.189206] LustreError: 169773:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3927.191844] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3927.872807] Lustre: server umount lustre-MDT0001 complete [ 3937.500308] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 20:51:28 (1767318688) [ 3949.453439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3949.467299] Lustre: *** cfs_fail_loc=197, val=0*** [ 3965.154380] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3970.272531] LustreError: 179291:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.107@tcp: failed processing log, type 4: rc = -110 [ 3975.444211] Lustre: *** cfs_fail_loc=197, val=0*** [ 3975.448919] Lustre: Skipped 65 previous similar messages [ 3996.001926] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3996.020295] Lustre: Skipped 4 previous similar messages [ 4004.149869] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 4011.595478] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4013.797892] Lustre: server umount lustre-OST0000 complete [ 4023.666480] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 20:52:55 (1767318775) [ 4061.421364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4066.285323] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 4076.051552] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4081.765691] LustreError: 181556:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 4081.779870] LustreError: 181556:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 4081.791884] LustreError: 181556:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 4082.670648] Lustre: server umount lustre-MDT0000 complete [ 4098.438153] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4135.154686] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 4212.493435] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 4233.292952] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4233.960159] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4236.131579] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4238.284034] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4256.436960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4256.598784] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4256.973958] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4257.053644] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4257.247612] Lustre: lustre-MDT0000: new disk, initializing [ 4257.522943] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4263.025121] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 4286.938865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4287.111520] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4287.343779] Lustre: 192101:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4287.728759] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4293.308887] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 4304.150324] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4308.370538] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4323.050827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4323.129598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4330.235836] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4330.424329] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 4338.972692] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4343.930260] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4344.142713] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4349.325447] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4349.532409] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4356.070035] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4356.071631] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4356.079854] LustreError: Skipped 3 previous similar messages [ 4356.080517] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4356.080521] Lustre: Skipped 13 previous similar messages [ 4356.140683] Lustre: Skipped 5 previous similar messages [ 4357.712819] LustreError: 195739:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4357.721941] LustreError: 195739:0:(obd_class.h:479:obd_check_dev()) Skipped 41 previous similar messages [ 4358.055906] Lustre: server umount lustre-OST0000 complete [ 4369.410774] Lustre: server umount lustre-MDT0000 complete [ 4371.442895] LustreError: 192115:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4371.458907] LustreError: 192115:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 4374.320291] LustreError: 192095:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767319127 with bad export cookie 2105318964584168318 [ 4374.323633] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4374.336170] LustreError: 192095:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4374.709775] Lustre: server umount lustre-MDT0001 complete [ 4383.936433] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 20:58:55 (1767319135) [ 4394.488405] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 4398.210437] Key type lgssc unregistered [ 4398.510810] LNet: 197014:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4398.519359] LNetError: 197014:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4398.541656] LNet: Removed LNI 192.168.203.107@tcp [ 4399.645672] Key type .llcrypt unregistered [ 4399.648867] Key type ._llcrypt unregistered [ 4433.720823] Key type ._llcrypt registered [ 4433.723789] Key type .llcrypt registered [ 4433.839638] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 4512.200261] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 4513.988200] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4514.003429] alg: No test for adler32 (adler32-zlib) [ 4515.060666] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 4515.526899] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 4517.288564] Key type lgssc registered [ 4518.422676] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4538.832297] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4539.384315] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4540.252979] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4543.326548] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4559.019082] Lustre: DEBUG MARKER: conf-sanity test_87: @@@@@@ FAIL: add mds1 failed [ 4565.286725] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 21:01:56 (1767319316) [ 4638.342784] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 4656.561360] Lustre: DEBUG MARKER: conf-sanity test_88: @@@@@@ FAIL: add mds1 failed [ 4662.816123] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 21:03:34 (1767319414) [ 4677.558570] Lustre: DEBUG MARKER: conf-sanity test_89: @@@@@@ FAIL: tunefs --param failover.node=192.0.2.254@tcp0 failed [ 4683.577385] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 21:03:55 (1767319435) [ 4698.442911] Lustre: DEBUG MARKER: conf-sanity test_90a: @@@@@@ FAIL: MDT start failed [ 4704.333592] Lustre: DEBUG MARKER: == conf-sanity test 90b: check max_mod_rpcs_in_flight is enforced after update ========================================================== 21:04:15 (1767319455) [ 4720.050871] Lustre: DEBUG MARKER: conf-sanity test_90b: @@@@@@ FAIL: MDT start failed [ 4725.706638] Lustre: DEBUG MARKER: == conf-sanity test 90c: check max_mod_rpcs_in_flight update limits ========================================================== 21:04:36 (1767319476) [ 4741.263738] Lustre: DEBUG MARKER: conf-sanity test_90c: @@@@@@ FAIL: MDT start failed [ 4746.736994] Lustre: DEBUG MARKER: == conf-sanity test 90d: check one close RPC is allowed above max_mod_rpcs_in_flight ========================================================== 21:04:58 (1767319498) [ 4762.540806] Lustre: DEBUG MARKER: conf-sanity test_90d: @@@@@@ FAIL: MDT start failed [ 4769.791965] Lustre: DEBUG MARKER: == conf-sanity test 91: evict-by-nid support ============= 21:05:20 (1767319520) [ 4785.312811] Lustre: DEBUG MARKER: conf-sanity test_91: @@@@@@ FAIL: MDS start failed [ 4791.714759] Lustre: DEBUG MARKER: == conf-sanity test 92: ldev returns MGS NID correctly in command substitution ========================================================== 21:05:42 (1767319542) [ 4815.266823] Lustre: DEBUG MARKER: == conf-sanity test 93: register mulitple MDT at the same time ========================================================== 21:06:06 (1767319566) [ 4817.250515] Lustre: DEBUG MARKER: SKIP: conf-sanity test_93 needs >= 3 MDTs [ 4819.631309] Lustre: DEBUG MARKER: == conf-sanity test 94: ldev outputs correct labels for file system name query ========================================================== 21:06:10 (1767319570) [ 4839.176291] Lustre: DEBUG MARKER: == conf-sanity test 95: ldev should only allow one label filter ========================================================== 21:06:30 (1767319590) [ 4858.448470] Lustre: DEBUG MARKER: == conf-sanity test 96: ldev returns hostname and backend fs correctly in command sub ========================================================== 21:06:49 (1767319609) [ 4878.367317] Lustre: DEBUG MARKER: == conf-sanity test 97: ldev returns correct ouput when querying based on role ========================================================== 21:07:09 (1767319629) [ 4899.861642] Lustre: DEBUG MARKER: == conf-sanity test 98: Buffer-overflow check while parsing mount_opts ========================================================== 21:07:31 (1767319651) [ 4913.898061] Lustre: DEBUG MARKER: conf-sanity test_98: @@@@@@ FAIL: MDT start failed [ 4918.993762] Lustre: DEBUG MARKER: == conf-sanity test 99: Adding meta_bg option ============ 21:07:50 (1767319670) [ 4997.910753] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 5008.750984] Lustre: DEBUG MARKER: conf-sanity test_99: @@@@@@ FAIL: add ost1 failed with new params [ 5014.619309] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 21:09:25 (1767319765) [ 5038.127714] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 5115.084484] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 5138.097940] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5138.320737] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5139.231230] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5142.849590] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5160.989417] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5161.073903] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5161.093792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5162.399911] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5162.454227] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5162.556494] Lustre: lustre-MDT0000: new disk, initializing [ 5162.693561] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5162.719336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5167.721346] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5189.481811] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5189.531730] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5189.677825] Lustre: 231607:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5189.758040] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5189.761071] Lustre: Skipped 1 previous similar message [ 5189.815084] Lustre: lustre-MDT0001: new disk, initializing [ 5189.896307] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5189.949882] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5189.966098] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5195.720628] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5206.549684] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5210.970519] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5227.195752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5227.325934] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5227.757416] Lustre: lustre-OST0000: new disk, initializing [ 5227.770494] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5227.894363] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5235.357479] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5235.370592] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5235.557212] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5235.641909] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5244.194378] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5256.168330] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5256.173950] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5256.181227] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5256.212753] Lustre: Skipped 1 previous similar message [ 5261.288366] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5261.298562] Lustre: Skipped 2 previous similar messages [ 5262.156171] LustreError: 234887:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5262.256104] Lustre: server umount lustre-OST0000 complete [ 5271.522434] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5271.549711] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5271.552384] Lustre: Skipped 1 previous similar message [ 5273.884430] LustreError: 235138:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 5273.903668] LustreError: 235138:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5274.029673] Lustre: server umount lustre-MDT0000 complete [ 5279.613908] LustreError: 231601:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767320033 with bad export cookie 14421995611054726354 [ 5279.622345] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5279.628403] LustreError: 231601:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5279.842250] LustreError: 235388:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5279.853737] LustreError: 235388:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5280.117730] Lustre: server umount lustre-MDT0001 complete [ 5295.940886] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 5299.433974] Key type lgssc unregistered [ 5299.838429] LNet: 236064:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5299.847380] LNetError: 236064:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5299.862753] LNet: Removed LNI 192.168.203.107@tcp [ 5300.760825] Key type .llcrypt unregistered [ 5300.768031] Key type ._llcrypt unregistered [ 5391.424344] Key type ._llcrypt registered [ 5391.426316] Key type .llcrypt registered [ 5391.539540] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 5392.650528] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5392.724093] alg: No test for adler32 (adler32-zlib) [ 5393.798672] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 5394.082062] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 5395.784438] Key type lgssc registered [ 5397.012934] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5405.983524] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5406.000253] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5407.553333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5412.666387] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5417.962308] LustreError: 240523:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5417.973903] LustreError: 240523:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5423.081497] LustreError: 240523:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5427.416704] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5427.935382] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5432.582060] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5440.162466] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5444.446428] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5457.536893] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5458.163042] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5465.670931] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 5473.438932] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5477.652574] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5486.218796] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5486.241958] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5486.273099] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5486.286286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.305971] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5487.405522] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5487.425688] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5487.438249] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5487.452992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.475398] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5488.828883] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5488.850542] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5488.873622] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5488.885147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.912492] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5490.081348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.986140] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5490.991339] Lustre: Skipped 1 previous similar message [ 5491.033788] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5491.044222] Lustre: Skipped 1 previous similar message [ 5491.047977] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5491.060111] Lustre: Skipped 1 previous similar message [ 5491.063440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.074842] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5491.082073] Lustre: Skipped 1 previous similar message [ 5492.211267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.364571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.347051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.397947] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5495.409894] Lustre: Skipped 3 previous similar messages [ 5495.429142] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5495.448755] Lustre: Skipped 3 previous similar messages [ 5495.453657] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5495.459118] Lustre: Skipped 3 previous similar messages [ 5495.468366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.480120] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5495.490022] Lustre: Skipped 3 previous similar messages [ 5496.631458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.814960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.914709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.901285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.251935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.311819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.286668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.241766] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5504.245536] Lustre: Skipped 7 previous similar messages [ 5504.274695] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5504.284581] Lustre: Skipped 7 previous similar messages [ 5504.294435] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5504.300539] Lustre: Skipped 7 previous similar messages [ 5504.304694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.316368] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5504.321212] Lustre: Skipped 7 previous similar messages [ 5505.540898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.753579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.993817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.294382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.308830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.283029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.381502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.496893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.486540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.448989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.421347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.429960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.186828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.350056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.373047] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5520.376532] Lustre: Skipped 14 previous similar messages [ 5520.389454] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5520.395854] Lustre: Skipped 14 previous similar messages [ 5520.400981] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5520.416465] Lustre: Skipped 14 previous similar messages [ 5520.420616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.428388] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5520.434498] Lustre: Skipped 14 previous similar messages [ 5521.521540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.484410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.714245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.696456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.855734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.874529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.964715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.143600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.210785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.260148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.172111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.220904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.243275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.187160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.271729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.410456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.529851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.675878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.565714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.557847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.473513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.517227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.266994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.165510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.022973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.870626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.826400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.938958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.864434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.912969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.262687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.434839] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5553.449810] Lustre: Skipped 31 previous similar messages [ 5553.475094] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5553.481389] Lustre: Skipped 31 previous similar messages [ 5553.490387] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5553.493023] Lustre: Skipped 31 previous similar messages [ 5553.499334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.514453] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5553.517253] Lustre: Skipped 31 previous similar messages [ 5554.676406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.909216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.046967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.934243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.075537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.261878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.180676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.466930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.416305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.538878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.877500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.176275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.467907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.876618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.980373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.102434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.121225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.277941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.181348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.255904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.348229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.483903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.758924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.786863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.926914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.926127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.071187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.068316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.070112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.300861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.450508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.723375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.083484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.170105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.159390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.144570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.154365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.187879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.270490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.321056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.373443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.564381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.686442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.894612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.939875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.023241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.392094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.650286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.773727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.833572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.801830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.864056] LustreError: 240542:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5611.879949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.881381] LustreError: 240542:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5612.925649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.942286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.137649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.163761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.260504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.403733] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5618.411368] Lustre: Skipped 57 previous similar messages [ 5618.443909] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5618.460864] Lustre: Skipped 57 previous similar messages [ 5618.465781] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5618.473823] Lustre: Skipped 57 previous similar messages [ 5618.478958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.497723] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5618.509263] Lustre: Skipped 57 previous similar messages [ 5619.501265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.557844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.614718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.620930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.748855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.962744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.170091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.406494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.624923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.790490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.896123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.796921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.863487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.902563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.032967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.181589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.311515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.266189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.314660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.359744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.455990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.508797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.505393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.397195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.536066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.756463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.011221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.303396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.684833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.652249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.664079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.757578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.289960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.529033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.733538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.891782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.116742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.256148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.368232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.767534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.933976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.978154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.496550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.028313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.087546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.338066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.509617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.495419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.906896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.140360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.430210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.939485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.192880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.482520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.741784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.730072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.744707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.799252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.221156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.132478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.258683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.251585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.328569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.143563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.155191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.039208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.131036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.154493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.237313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.148607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.109806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.212238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.261938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.311892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.340737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.499755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.590479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.654908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.826817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.727563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.706837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.676929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.520884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.401290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.344875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.280538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.182684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.045947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.972538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.937958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.881518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.722268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.835796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.798888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.908494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.769679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.622819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.615802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.709557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.624227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.817072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.637690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.694617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.520298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.482916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.715630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.638881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.588812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.792899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.929922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.787423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.857987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.878772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.888713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.768947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.335699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.349631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.144741] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5747.150776] Lustre: Skipped 117 previous similar messages [ 5747.162229] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5747.171352] Lustre: Skipped 117 previous similar messages [ 5747.175831] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5747.178481] Lustre: Skipped 117 previous similar messages [ 5747.183466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.202594] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5747.205879] Lustre: Skipped 117 previous similar messages [ 5748.231820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.944205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.998375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.609152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.470964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.588299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.816887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.977432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.037112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.230238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.319509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.544362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.618861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.705530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.811519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.752668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.736131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.730618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.866611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.932765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.002184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.062762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.116343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.076258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.070308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.208317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.153668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.234660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.161412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.099774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.192900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.323563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.227882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.407714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.668752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.750698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.935688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.002428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.227740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.310731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.513442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.512078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.762103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.938914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.766319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.922760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.846023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.920899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.975324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.887329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.750312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.705717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.755949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.674676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.692357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.697725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.903042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.911541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.021495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.976846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.087709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.005975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.181238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.186438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.187128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.391370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.598250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.839898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.910819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.989623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.259066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.515310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.697972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.647416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.944258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.863724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.911578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.150822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.181894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.536008] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.836768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.903355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.947231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.898187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.017180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.875913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.771167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.750893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.700679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.497983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.503695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.382286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.427691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.287789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.282645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.194505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.164509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.119630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.231444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.088416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.115843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.863492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.757322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.865317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.817211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.873627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.855502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.761059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.796582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.738381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.011483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.967947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.085512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.000327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.017966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.066153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.142721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.007708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.246108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.310222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.603570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.729606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.997602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.246476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.334677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.521636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.701058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.741957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.765979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.735878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.719986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.569943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.545444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.600496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.733081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.859035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.678921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.667632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.631802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.568606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.475292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.565916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.581205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.642295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.578067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.576795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.431425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.662730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.597842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.396255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.235119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.185823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.141135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.351654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.273894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.368057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.368980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.321903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.259936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.162025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.186321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.208512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.152808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.132493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.116947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.184896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.186680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.174400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.180330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.396660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.364722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.347734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.385605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.198903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.146593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.031899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.028857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.963175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.072250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.016434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.046780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.970825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.056473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.039617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.103931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.981912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.061760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.897662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.739745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.729752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.707030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.593690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.463919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.435123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.329927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.411943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.445236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.416929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.224087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.146092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.025036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.195420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.254904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.381689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.178043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.378111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.261648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.145040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.151795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.131363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.980737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.935511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.877393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.925812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.839062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.784665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.882293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.823664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.700905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.766839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.677529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.417518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.212327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.186527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.379747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.371404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.329069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.364710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.426446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.334718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.443689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.586507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.556340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.544219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.591330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.630869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.595547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.630638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.578463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.489890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.402408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.469757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.453203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.487326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.514344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.500124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.427660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.533779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.336808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.256393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.215602] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6003.221569] Lustre: Skipped 250 previous similar messages [ 6003.242932] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6003.253157] Lustre: Skipped 250 previous similar messages [ 6003.261411] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6003.264232] Lustre: Skipped 250 previous similar messages [ 6003.270483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.290413] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6003.299128] Lustre: Skipped 250 previous similar messages [ 6004.239575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.127886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.104085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.936346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.782278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.650206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.628359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.413493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.423655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.574236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.433748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.483386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.520337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.463528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.479348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.429814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.378180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.213828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.289379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.427287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.375803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.307173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.179588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.156881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.988078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.223034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.317860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.227476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.257230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.399273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.664075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.636849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.692402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.674161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.799776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.887591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.032300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.020741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.024303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.042105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.940316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.714887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.709992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.719839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.904570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.925598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.920035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.889346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.967994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.861414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.692917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.735752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.602854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.535387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.346732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.239738] LustreError: 240542:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6058.248595] LustreError: 240542:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6058.255210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.023467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.084566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.097699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.151511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.097911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.032532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.034509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.172593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.328362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.330448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.479217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.146500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.912053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.529166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.471341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.184984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.091372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.002548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.708153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.720770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.650416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.780817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.713089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.222547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.548440] LustreError: 240542:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6082.568721] LustreError: 240542:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6082.580431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.820172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.715588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.714326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.676539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.768590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.737790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.814343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.860643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.014524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.195273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.946174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.239789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.190557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.153086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.969792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.887100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.043840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.933799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.021909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.043878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.109085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.049521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.891310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.990472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.831481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.838536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.754091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.766122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.571288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.732652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.662546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.713664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.590988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.493576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.671226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.526892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.392782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.144190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.214626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.126529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.134166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.181050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.282631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.291731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.286576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.182892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.177977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.993984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.011988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.173666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.193468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.288125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.286908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.591358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.685727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.031135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.281714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.400883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.651900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.668541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.704805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.575255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.499789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.280142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.234326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.316309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.073581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.926559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.728481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.798447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.600292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.478482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.354871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.285311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.191331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.066336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.927182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.952090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.755317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.733401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.718299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.800815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.502722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.342748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.115163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.984762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.700256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.500046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.577479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.533123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.524037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.319459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.118888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.941505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.929543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.673860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.557581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.506604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.400050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.110224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.801657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.486123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.247376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.111464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.089234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.884856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.944392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.795569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.680674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.485863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.255984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.219775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.070288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.945966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.058773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.093692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.195323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.224619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.118449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.959682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.966288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.791730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.860741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.705541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.636313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.483357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.451367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.622669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.717198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.788640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.760353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.812898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.874777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.860162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.844543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.846678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.880920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.774334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.744573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.654189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.480623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.450442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.371464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.174282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.931982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.777446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.668320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.528760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.384789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.388810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.411753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.454441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.446407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.558603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.544368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.522592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.526980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.460498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.609774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.789810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.668822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.751720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.782772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.719089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.628737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.584264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.564376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.397621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.243849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.019972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.966783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.968088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.919371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.888938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.893676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.894942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.715814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.713197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.782554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.880986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.871429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.920466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.966217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.800592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.742305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.731221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.843121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.786630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.758331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.735462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.929694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.946489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.945457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.972638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.081594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.139168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.085311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.036761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.296709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.297320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.313660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.254734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.249678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.198776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.133500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.149924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.169697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.258948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.469980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.904667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.917915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.940923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.718463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.752741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.683888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.580778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.480632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.546609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.611089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.482307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.466250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.322568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.291927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.158877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.221869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.158411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.308734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.435263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.541542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.486566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.553235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.667769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.806195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.885604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.021927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.883447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.824407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.432895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.968149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.935648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.832360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.590022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.306633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.175543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.285822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.989951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.959760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.897682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.226515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.238133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.343873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.184885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.042811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.887542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.854533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.773782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.768739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.654995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.753927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.764448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.835309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.692642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.734411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.639611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.704390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.604832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.701190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.732155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.611149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.572636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.696789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.475988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.591949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.515076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.745770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.594344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.647796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.485685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.482325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.615213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.777873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.713205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.853923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.753409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.638931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.590470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.637161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.548989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.554958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.465582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.381903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.251416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.998965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.813403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.851538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.716826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.620575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.475051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.389814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.367221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.384589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.332983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.531670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.429634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.415233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.285581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.412896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.528431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.565505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.583669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.555845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.597966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.565469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.495564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.282608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.264390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.325044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.218453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.321245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.189224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.831732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.681451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.675711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.478677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.428687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.171709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.274907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.148470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.068328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.093129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.147448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.114906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.039841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.902520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.029924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.215723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.295436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.375619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.178288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.193410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.362845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.362582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.210955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.040232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.051933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.895444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.072667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.966115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.984408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.698512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.614666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.352228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.135397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.830159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.836050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.643597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.534837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.592455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.492345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.278663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.133437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.992452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.956791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.807342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.716376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.668636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.583522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.425826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.404162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.174573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.172257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.969716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.033410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.967833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.929321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.749553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.626954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.463958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.333571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.349630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.351349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.551767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.836974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.070234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.014626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.107259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.226561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.168385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.349733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.329557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.469145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.325247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.237836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.120636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.141516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.051693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.169674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.919639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.914100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.942062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.077278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.081820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.085946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.056175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.993961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.980314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.961904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.061568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.910687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.746661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.751443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.672845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.813690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.690165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.902569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.822746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.891031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.898957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.866839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.803695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.970638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.096218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.095664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.051442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.784473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.561870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.507811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.125255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.963038] LustreError: 242511:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6883.973334] LustreError: 240542:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7515.617948] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7515.623118] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7515.631311] Lustre: Skipped 509 previous similar messages [ 7515.634460] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7517.676393] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7517.685664] Lustre: Skipped 1 previous similar message [ 7520.826865] LustreError: 294904:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7521.088134] Lustre: server umount lustre-OST0000 complete [ 7527.393827] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7527.409863] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7533.029576] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7533.033590] Lustre: Skipped 2 previous similar messages [ 7538.145404] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7538.151650] Lustre: Skipped 1 previous similar message [ 7540.192506] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7540.310222] LustreError: 295154:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7540.318301] LustreError: 295154:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7540.584676] Lustre: server umount lustre-MDT0000 complete [ 7543.267561] LustreError: 240518:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7543.282913] LustreError: 240518:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7545.012151] LustreError: 240503:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767322298 with bad export cookie 14181639999156332021 [ 7545.030163] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7545.032804] LustreError: 240503:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7545.185576] LustreError: 295407:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7545.193758] LustreError: 295407:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7545.379069] Lustre: server umount lustre-MDT0001 complete [ 7557.014774] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 7559.341322] Key type lgssc unregistered [ 7559.571839] LNet: 296085:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7559.580406] LNetError: 296085:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7559.598931] LNet: Removed LNI 192.168.203.107@tcp [ 7560.292987] Key type .llcrypt unregistered [ 7560.295157] Key type ._llcrypt unregistered [ 7640.580828] Key type ._llcrypt registered [ 7640.581970] Key type .llcrypt registered [ 7640.652933] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 7641.781822] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7641.813971] alg: No test for adler32 (adler32-zlib) [ 7642.844189] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 7643.034347] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 7644.730418] Key type lgssc registered [ 7645.848446] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7653.706953] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7653.725366] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7655.112794] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7658.743926] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7664.615573] LustreError: 300539:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7664.636558] LustreError: 300539:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7669.400743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7669.673368] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7673.595463] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7678.456062] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7681.746295] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7691.516788] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7691.803491] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7697.518668] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7700.022742] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 7702.926799] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7707.869641] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7709.664042] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7709.674687] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7709.686344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7709.699195] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7709.700050] LustreError: 300559:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7714.736660] Lustre: lustre-OST0000: Not available for connect from 192.168.203.7@tcp (stopping) [ 7715.300209] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7715.311902] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7715.320412] Lustre: Skipped 1 previous similar message [ 7715.331872] Lustre: Skipped 1 previous similar message [ 7717.694640] LustreError: 303321:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7717.868991] Lustre: server umount lustre-OST0000 complete [ 7729.800232] LustreError: 300559:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7756.243503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7756.635753] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7762.998493] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7768.536256] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7780.148631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7780.162605] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7780.171419] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7780.184219] Lustre: Skipped 1 previous similar message [ 7781.803902] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 11 sec [ 7785.217983] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff99705129a000.ost_server_uuid 50 [ 7786.546430] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff99705129a000.ost_server_uuid in FULL state after 0 sec [ 7790.565401] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7790.577085] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7790.586110] Lustre: Skipped 1 previous similar message [ 7790.602155] Lustre: Skipped 1 previous similar message [ 7795.650128] LustreError: 304859:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7795.657228] LustreError: 304859:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7795.686206] LustreError: 303995:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7795.699964] LustreError: 303995:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7795.761233] Lustre: server umount lustre-OST0000 complete [ 7800.802919] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7800.810624] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7800.815288] Lustre: Skipped 1 previous similar message [ 7805.394047] LustreError: 305109:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7805.396329] LustreError: 305109:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7805.463410] Lustre: server umount lustre-MDT0000 complete [ 7805.928734] LustreError: 300539:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7805.940445] LustreError: 300539:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7809.623385] LustreError: 304017:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767322563 with bad export cookie 1285940882508659568 [ 7809.626829] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7809.628452] LustreError: 304017:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7809.768995] LustreError: 305359:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7809.770991] LustreError: 305359:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7809.937845] Lustre: server umount lustre-MDT0001 complete [ 7821.687138] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 7824.400298] Key type lgssc unregistered [ 7824.778135] LNet: 306035:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7824.795160] LNetError: 306035:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7824.820588] LNet: Removed LNI 192.168.203.107@tcp [ 7825.567833] Key type .llcrypt unregistered [ 7825.571863] Key type ._llcrypt unregistered [ 7909.338428] Key type ._llcrypt registered [ 7909.340387] Key type .llcrypt registered [ 7909.443952] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 7910.233647] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7910.398212] alg: No test for adler32 (adler32-zlib) [ 7911.438147] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 7911.614340] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 7913.257108] Key type lgssc registered [ 7914.363580] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7929.317058] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7929.345917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7929.399326] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7930.480143] LustreError: 310018:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000006d319e0b x1853172130775552/t0(0) o250->MGC192.168.203.107@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7930.657533] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7934.767119] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7935.137524] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7940.002325] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7940.291251] Lustre: 312722:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7949.991827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7950.401218] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7950.407228] Lustre: Skipped 1 previous similar message [ 7954.570975] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7954.654881] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7954.792995] Lustre: lustre-OST0001: new disk, initializing [ 7954.795950] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7954.929801] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7955.524318] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7955.961124] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 7959.748673] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 7960.171534] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7960.178690] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7960.200069] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000400 [ 7970.846526] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7976.903281] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7995.130911] LustreError: 315856:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 7995.212915] Lustre: server umount lustre-MDT0000 complete [ 7996.387277] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7996.388955] LustreError: 315388:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7996.389150] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7996.405527] Lustre: Skipped 2 previous similar messages [ 7996.429358] LustreError: 315388:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 7999.194531] LustreError: 315877:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 7999.197985] LustreError: 315877:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7999.291496] Lustre: server umount lustre-MDT0001 complete [ 8005.640740] LustreError: 312686:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767322759 with bad export cookie 4574446412884450881 [ 8005.651536] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8005.652894] LustreError: 312686:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8005.708416] LustreError: 316444:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8005.712655] LustreError: 316444:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8005.740529] Lustre: server umount lustre-OST0000 complete [ 8005.945397] Lustre: server umount lustre-OST0001 complete [ 8013.431921] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 8013.507291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8015.099923] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 8015.146467] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8016.662529] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 8016.763953] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8019.552492] Lustre: DEBUG MARKER: conf-sanity test_103: @@@@@@ FAIL: (9) Fail to rename OST 2 [ 8026.057254] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 21:59:37 (1767322777) [ 8047.747468] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 8097.374335] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 8113.653160] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8114.055628] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8114.611929] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8116.744632] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8127.531320] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8127.570713] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8127.570713] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8127.570713] [ 8127.580276] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8127.715155] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8127.739979] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8127.795309] Lustre: lustre-MDT0000: new disk, initializing [ 8127.855154] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8127.867899] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8130.448227] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8141.366264] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8141.392949] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8141.454924] Lustre: 326474:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8141.512656] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8141.515431] Lustre: Skipped 1 previous similar message [ 8141.565972] Lustre: lustre-MDT0001: new disk, initializing [ 8141.637763] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8141.660911] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8141.668288] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8144.373354] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8156.287988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8156.335223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8156.501176] Lustre: lustre-OST0000: new disk, initializing [ 8156.504965] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8156.587521] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8157.666427] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8157.674611] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8157.752353] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8161.442951] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8175.911530] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8175.987912] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8176.087788] Lustre: lustre-OST0001: new disk, initializing [ 8176.091184] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8176.180291] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8178.004735] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8178.011803] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8178.139743] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8181.358392] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8190.076510] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 22:02:22 (1767322942) [ 8200.976811] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 22:02:32 (1767322952) [ 8219.107499] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8219.111713] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8219.119960] Lustre: Skipped 1 previous similar message [ 8219.127706] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8219.130319] Lustre: Skipped 2 previous similar messages [ 8220.465299] LustreError: 331124:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8220.468176] LustreError: 331124:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 8220.490970] Lustre: server umount lustre-OST0000 complete [ 8223.714269] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8223.721299] LustreError: Skipped 1 previous similar message [ 8223.723992] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8223.733876] Lustre: Skipped 1 previous similar message [ 8223.738781] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8224.230882] LustreError: 328703:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8229.350124] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8229.353494] LustreError: 328702:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8229.355187] Lustre: Skipped 5 previous similar messages [ 8229.417831] LustreError: 331373:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 8229.421251] LustreError: 331373:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8229.577972] Lustre: server umount lustre-MDT0000 complete [ 8232.810917] LustreError: 326466:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767322986 with bad export cookie 4574446412884453667 [ 8232.813829] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8232.818641] LustreError: 326466:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8234.465965] LustreError: 326866:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8234.471594] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8234.486050] Lustre: Skipped 2 previous similar messages [ 8234.489949] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8242.595062] LustreError: 328799:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8242.597510] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8242.615258] LustreError: 328799:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8242.622874] Lustre: Skipped 1 previous similar message [ 8247.264333] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8247.333502] LustreError: 331623:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 8247.336992] LustreError: 331623:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8247.424128] Lustre: server umount lustre-MDT0001 complete [ 8256.063598] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 8270.202656] LustreError: 333210:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8270.208710] LustreError: 333210:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8270.302726] Lustre: server umount lustre-OST0001 complete [ 8282.893534] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 8332.597824] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 8345.825101] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8346.250757] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8347.649876] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8348.977967] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8360.878355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8360.950433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8361.232302] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8361.249975] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8361.308953] Lustre: lustre-MDT0000: new disk, initializing [ 8361.403636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8361.419346] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8364.283668] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8377.540803] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8377.572968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8377.623762] Lustre: 341037:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8377.673224] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8377.675903] Lustre: Skipped 1 previous similar message [ 8377.723401] Lustre: lustre-MDT0001: new disk, initializing [ 8377.846060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8377.850248] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8380.888406] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8388.653526] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8391.878846] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8401.514902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8401.593036] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8403.041297] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8403.140483] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8406.196789] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8412.556638] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8418.790919] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8418.794069] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8418.806858] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8418.828690] LustreError: Skipped 1 previous similar message [ 8423.747429] LustreError: 344170:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8423.758885] LustreError: 344170:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8423.931078] Lustre: server umount lustre-OST0000 complete [ 8429.025631] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8429.048177] Lustre: Skipped 1 previous similar message [ 8442.336151] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8442.463613] Lustre: server umount lustre-MDT0000 complete [ 8444.399650] LustreError: 341044:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8444.433338] LustreError: 341044:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8446.637497] LustreError: 341030:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767323200 with bad export cookie 4574446412884455130 [ 8446.640778] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8446.649961] LustreError: 341030:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8446.945292] Lustre: server umount lustre-MDT0001 complete [ 8457.555130] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 8459.836915] Key type lgssc unregistered [ 8460.111759] LNet: 345347:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8460.117795] LNetError: 345347:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8460.133583] LNet: Removed LNI 192.168.203.107@tcp [ 8460.774177] Key type .llcrypt unregistered [ 8460.778428] Key type ._llcrypt unregistered [ 8522.708937] Key type ._llcrypt registered [ 8522.711319] Key type .llcrypt registered [ 8522.776313] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 8523.687035] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8523.700880] alg: No test for adler32 (adler32-zlib) [ 8524.687878] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 8524.903359] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 8526.552157] Key type lgssc registered [ 8527.337820] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8532.511336] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8532.523481] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8533.955781] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8537.198691] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8544.225265] LustreError: 349993:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8544.240693] LustreError: 349993:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8546.497996] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8546.749115] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8549.817297] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8553.725645] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8555.935629] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8563.024377] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8563.275759] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8567.356259] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8572.134942] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8572.965835] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8576.994900] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8576.997468] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8576.999305] LustreError: Skipped 1 previous similar message [ 8577.025801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8579.107459] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8580.661785] LustreError: 352669:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8580.767858] Lustre: server umount lustre-OST0000 complete [ 8587.745803] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8587.751191] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8587.760526] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8587.768121] Lustre: Skipped 1 previous similar message [ 8589.283232] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8590.512589] LustreError: 352923:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 8590.528383] LustreError: 352923:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8590.621371] Lustre: server umount lustre-MDT0000 complete [ 8594.170676] LustreError: 349971:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767323347 with bad export cookie 5442160987721708092 [ 8594.177473] LustreError: 350897:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0001: can't precreate: rc = -5 [ 8594.191744] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8594.201517] LustreError: 350897:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0001: cannot precreate objects: rc = -5 [ 8594.299793] LustreError: 353172:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8594.311701] LustreError: 353172:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8594.445268] Lustre: server umount lustre-MDT0001 complete [ 8603.755409] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 8605.588840] Key type lgssc unregistered [ 8605.872726] LNet: 353848:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8605.881636] LNetError: 353848:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8605.902668] LNet: Removed LNI 192.168.203.107@tcp [ 8606.514354] Key type .llcrypt unregistered [ 8606.517271] Key type ._llcrypt unregistered [ 8653.776700] Key type ._llcrypt registered [ 8653.778288] Key type .llcrypt registered [ 8653.836218] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 8654.700719] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8654.715037] alg: No test for adler32 (adler32-zlib) [ 8655.675260] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 8655.812285] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 8657.432625] Key type lgssc registered [ 8658.190273] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8666.652841] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8666.668431] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8682.144431] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8687.264558] LustreError: 358078:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.107@tcp: failed processing log, type 4: rc = -110 [ 8713.952210] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8717.498225] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8721.142586] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8722.345703] LustreError: 358743:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 8722.414393] Lustre: server umount lustre-OST0000 complete [ 8735.695100] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 8737.340745] Key type lgssc unregistered [ 8737.529895] LNet: 359820:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8737.535545] LNetError: 359820:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8738.599814] LNet: Removed LNI 192.168.203.107@tcp [ 8739.075592] Key type .llcrypt unregistered [ 8739.078435] Key type ._llcrypt unregistered [ 8793.671640] Key type ._llcrypt registered [ 8793.675295] Key type .llcrypt registered [ 8793.739574] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 8794.566297] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8794.597957] alg: No test for adler32 (adler32-zlib) [ 8795.559864] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 8795.687906] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 8797.304290] Key type lgssc registered [ 8797.898963] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8805.598907] loop0: detected capacity change from 0 to 204800000 [ 8805.721942] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8805.727177] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8805.876292] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8805.918649] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8806.527273] loop0: detected capacity change from 0 to 204800000 [ 8806.615254] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8806.627507] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8806.791592] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8806.825203] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8807.420975] loop0: detected capacity change from 0 to 204800000 [ 8807.492527] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8807.496923] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8807.567930] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8807.626735] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8808.216294] loop0: detected capacity change from 0 to 204800000 [ 8808.314545] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8808.507145] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8809.109614] loop0: detected capacity change from 0 to 204800000 [ 8809.138094] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8811.542953] loop0: detected capacity change from 0 to 204800000 [ 8811.577807] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8813.967195] loop0: detected capacity change from 0 to 204800000 [ 8813.987785] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8816.570809] loop0: detected capacity change from 0 to 204800000 [ 8816.613387] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8819.251145] loop0: detected capacity change from 0 to 204800000 [ 8819.316501] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8819.371397] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8819.390279] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8819.443929] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8819.451655] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8819.460413] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8819.468348] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8819.481345] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8819.489187] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8819.496817] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8819.504192] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8819.512555] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8819.529528] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8819.537781] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8819.545267] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8819.555991] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8819.564252] Lustre: 366368:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8823.610875] LustreError: 366636:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8823.711629] Lustre: server umount MGS complete [ 8824.254396] loop0: detected capacity change from 0 to 204800000 [ 8824.318337] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8824.593294] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8824.630944] Lustre: lustre-MDT0000: Aborting client recovery [ 8825.203398] loop1: detected capacity change from 0 to 204800000 [ 8825.283349] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8825.336677] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8825.371735] Lustre: 366805:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8825.379820] Lustre: 366805:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8825.384939] Lustre: 366805:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8825.389829] Lustre: 366805:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8825.394182] Lustre: 366805:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8825.398095] Lustre: 366805:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8825.403292] Lustre: 366805:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8825.414744] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8825.526571] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8825.562803] Lustre: lustre-MDT0001: Aborting client recovery [ 8826.108853] loop2: detected capacity change from 0 to 204800000 [ 8826.211048] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8826.271694] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8826.301732] Lustre: 366887:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8826.308677] Lustre: 366887:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8826.316741] Lustre: 366887:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8826.323961] Lustre: 366887:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8826.332385] Lustre: 366887:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8826.340734] Lustre: 366887:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8826.345769] Lustre: 366887:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8826.418264] Lustre: Found index 0 for lustre-OST0000, updating log [ 8826.480634] Lustre: lustre-OST0000: Aborting recovery [ 8827.049885] loop3: detected capacity change from 0 to 204800000 [ 8827.123715] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8827.170835] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8827.207742] Lustre: 366978:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8827.216231] Lustre: 366978:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8827.226603] Lustre: 366978:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8827.232395] Lustre: 366978:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8827.237174] Lustre: 366978:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8827.241913] Lustre: 366978:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8827.246347] Lustre: 366978:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8827.304475] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8827.307956] Lustre: Skipped 1 previous similar message [ 8827.328297] Lustre: lustre-OST0001: Aborting recovery [ 8832.526590] LustreError: 366748:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 8834.538301] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8834.539146] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8834.567642] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8834.572122] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8835.558147] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8835.560719] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8840.088678] LustreError: 366716:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 280us: rc = -2 [ 8856.035718] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8856.037152] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8856.053742] Lustre: Skipped 3 previous similar messages [ 8856.064717] Lustre: Skipped 3 previous similar messages [ 8861.154826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8861.159730] Lustre: Skipped 3 previous similar messages [ 8866.272118] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8866.273597] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8866.288441] Lustre: Skipped 3 previous similar messages [ 8866.353345] LustreError: 367334:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 8866.356929] LustreError: 367334:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 8866.432909] Lustre: server umount lustre-MDT0000 complete [ 8867.007613] LustreError: 366909:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767323620 with bad export cookie 17914474448179966836 [ 8867.012982] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8867.015873] LustreError: 366909:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8871.393660] LustreError: 366712:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8871.394075] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8871.398101] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8871.413725] LustreError: 366712:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8871.426780] Lustre: Skipped 1 previous similar message [ 8872.417492] LustreError: 366714:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8872.425552] LustreError: 366714:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8873.182327] LustreError: 367387:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 8873.191366] LustreError: 367387:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8873.352859] Lustre: server umount lustre-MDT0001 complete [ 8888.288260] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8888.311089] LustreError: 367439:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 8888.314272] LustreError: 367439:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8888.353506] Lustre: server umount lustre-OST0000 complete [ 8903.137231] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8903.174356] LustreError: 367491:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8903.178163] LustreError: 367491:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8903.300861] Lustre: server umount lustre-OST0001 complete [ 8908.068818] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 22:14:20 (1767323660) [ 8928.843919] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 8962.645497] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 8971.272597] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8971.715598] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8972.762941] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8973.997363] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8981.753703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8981.789542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8981.908896] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8981.928779] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8981.970127] Lustre: lustre-MDT0000: new disk, initializing [ 8982.013000] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8982.030594] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8983.911989] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8992.039871] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8992.067684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8992.110617] Lustre: 377230:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8992.140672] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8992.143131] Lustre: Skipped 1 previous similar message [ 8992.183965] Lustre: lustre-MDT0001: new disk, initializing [ 8992.232195] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8992.248182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8992.252747] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8994.023234] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 8997.990509] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8999.874502] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9006.582435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9006.617550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9006.742736] Lustre: lustre-OST0000: new disk, initializing [ 9006.745464] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9006.783238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9009.467636] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9013.357837] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9015.810189] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9015.817152] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9015.883501] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9021.669345] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9026.132773] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9034.409426] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9072.099092] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9072.100208] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9072.102126] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9072.102131] Lustre: Skipped 3 previous similar messages [ 9072.104309] LustreError: Skipped 1 previous similar message [ 9072.107625] Lustre: Skipped 1 previous similar message [ 9082.342225] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9082.353166] Lustre: Skipped 4 previous similar messages [ 9084.384203] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9084.427468] LustreError: 381486:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 9084.429523] LustreError: 381486:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9084.507492] Lustre: server umount lustre-OST0000 complete [ 9087.457929] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9087.463407] Lustre: Skipped 1 previous similar message [ 9092.804116] Lustre: server umount lustre-MDT0000 complete [ 9095.018153] LustreError: 380414:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767323848 with bad export cookie 17914474448179969552 [ 9095.020569] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9095.025020] LustreError: 380414:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9095.177985] Lustre: server umount lustre-MDT0001 complete [ 9102.005161] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9103.744190] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9107.158083] LustreError: 383504:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9107.161311] LustreError: 383504:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 9107.190472] Lustre: server umount MGS complete [ 9115.234566] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9115.531347] LustreError: 384463:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9115.549544] LustreError: 384475:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9115.586530] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9117.486580] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9120.739094] LustreError: 384463:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9123.946945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9126.334488] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9129.125811] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9131.160044] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9136.850529] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9140.108371] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9143.003834] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9159.651945] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9159.653188] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9159.656646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9159.660393] Lustre: Skipped 1 previous similar message [ 9159.663528] Lustre: Skipped 5 previous similar messages [ 9163.118064] LustreError: 387502:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 9163.120486] LustreError: 387502:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 9163.166683] Lustre: server umount lustre-OST0000 complete [ 9165.281037] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9169.890341] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9169.894159] Lustre: Skipped 1 previous similar message [ 9171.357656] Lustre: server umount lustre-MDT0000 complete [ 9173.431703] LustreError: 384443:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767323926 with bad export cookie 17914474448179971379 [ 9173.435338] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9173.439039] LustreError: 384443:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9173.588479] Lustre: server umount lustre-MDT0001 complete [ 9179.789870] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 9181.255766] Key type lgssc unregistered [ 9181.442878] LNet: 388677:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9181.449894] LNetError: 388677:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9181.461741] LNet: Removed LNI 192.168.203.107@tcp [ 9181.852796] Key type .llcrypt unregistered [ 9181.854818] Key type ._llcrypt unregistered [ 9200.955771] Key type ._llcrypt registered [ 9200.957253] Key type .llcrypt registered [ 9201.008302] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 9232.626095] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 9233.097743] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9233.123349] alg: No test for adler32 (adler32-zlib) [ 9233.997754] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 9234.091305] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 9235.688425] Key type lgssc registered [ 9236.277118] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9244.870282] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9244.925904] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9245.820778] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9247.009125] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9254.568148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9254.600924] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9254.607726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9255.716099] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9255.731732] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9255.772847] Lustre: lustre-MDT0000: new disk, initializing [ 9255.814482] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9255.826227] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9257.514305] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9265.667147] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9265.708581] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9265.754548] Lustre: 398774:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9265.794093] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9265.798775] Lustre: Skipped 1 previous similar message [ 9265.841608] Lustre: lustre-MDT0001: new disk, initializing [ 9265.890587] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9265.906223] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9265.910629] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9267.900949] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9271.735400] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9273.563644] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9279.912440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9279.952856] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9280.092089] Lustre: lustre-OST0000: new disk, initializing [ 9280.094504] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9280.154895] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9282.819643] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9286.844162] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9288.708284] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9288.713412] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9288.771450] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9295.725422] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9306.589664] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9314.395607] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9324.247350] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9360.354928] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9360.357169] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9360.357244] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9360.361116] Lustre: Skipped 1 previous similar message [ 9362.413710] LustreError: 403276:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 9362.458551] Lustre: server umount lustre-OST0000 complete [ 9365.473598] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9365.474347] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9365.479290] Lustre: Skipped 1 previous similar message [ 9365.485043] Lustre: Skipped 1 previous similar message [ 9370.466175] LustreError: 403526:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 9370.471822] LustreError: 403526:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9370.540676] Lustre: server umount lustre-MDT0000 complete [ 9372.854611] LustreError: 402840:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767324126 with bad export cookie 1537795935251590519 [ 9372.858055] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9372.860101] LustreError: 402840:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9372.940021] LustreError: 403777:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 9372.945298] LustreError: 403777:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9373.027730] Lustre: server umount lustre-MDT0001 complete [ 9380.360492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9382.118852] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9386.022144] LustreError: 405294:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9386.024064] LustreError: 405294:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9386.056930] Lustre: server umount MGS complete [ 9393.661179] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9393.904073] LustreError: 406254:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9393.912264] LustreError: 406254:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9393.922413] LustreError: 406266:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9393.953294] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9395.737331] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9399.265036] LustreError: 406254:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9400.907648] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9401.087977] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9402.727682] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9405.029314] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9406.530935] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9410.969797] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9411.125991] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9413.661866] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9416.259808] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9444.833773] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9444.835371] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9444.836446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9444.840179] Lustre: Skipped 1 previous similar message [ 9444.843398] LustreError: Skipped 1 previous similar message [ 9449.953456] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9449.955822] Lustre: Skipped 1 previous similar message [ 9450.222237] LustreError: 409296:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 9450.224706] LustreError: 409296:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 9450.280344] Lustre: server umount lustre-OST0000 complete [ 9452.513259] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9452.516624] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9455.073975] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9455.078405] Lustre: Skipped 2 previous similar messages [ 9458.506724] LustreError: 409546:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 9458.508784] LustreError: 409546:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9458.539673] Lustre: server umount lustre-MDT0000 complete [ 9460.192971] LustreError: 407557:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9460.198690] LustreError: 407557:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9460.486868] LustreError: 406234:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767324214 with bad export cookie 1537795935251592458 [ 9460.489981] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9460.494020] LustreError: 406234:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9460.630043] Lustre: server umount lustre-MDT0001 complete [ 9466.254490] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [ 9467.565201] Key type lgssc unregistered [ 9467.703566] LNet: 410470:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9467.706439] LNetError: 410470:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9467.718348] LNet: Removed LNI 192.168.203.107@tcp [ 9468.076653] Key type .llcrypt unregistered [ 9468.078327] Key type ._llcrypt unregistered [ 9482.620213] Key type ._llcrypt registered [ 9482.621378] Key type .llcrypt registered [ 9482.664567] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 9502.744352] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 9503.237216] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9503.249283] alg: No test for adler32 (adler32-zlib) [ 9504.141061] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [ 9504.236625] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [ 9505.834029] Key type lgssc registered [ 9506.237783] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9513.318991] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9515.805878] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9534.788713] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 9541.268582] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9546.351763] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9546.375819] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9546.381503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9547.480971] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9547.496872] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9547.535711] Lustre: lustre-MDT0000: new disk, initializing [ 9547.564344] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9547.571859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9549.121989] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9557.015447] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9557.045520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9557.155086] Lustre: lustre-OST0000: new disk, initializing [ 9557.157656] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9557.159621] Lustre: Skipped 1 previous similar message [ 9557.189263] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9557.422601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9557.426295] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9557.449785] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9559.805818] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9895.394202] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9895.401143] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9897.282180] LustreError: 421670:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 9897.367069] Lustre: server umount lustre-MDT0000 complete [ 9898.880353] LustreError: 419476:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767324652 with bad export cookie 9635373942869277344 [ 9898.882544] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9898.885364] LustreError: 419476:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9904.943301] LustreError: 421945:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9904.945611] LustreError: 421945:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9904.997281] Lustre: server umount lustre-OST0000 complete [ 9920.730471] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [ 9948.246545] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [ 9955.536859] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9955.866523] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9956.480409] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9957.381462] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9963.535319] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9963.556662] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9963.652471] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9963.665111] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9963.701322] Lustre: lustre-MDT0000: new disk, initializing [ 9963.726365] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9963.733655] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9965.282523] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9972.124284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9972.151175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9972.189143] Lustre: 431032:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9972.214733] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9972.216737] Lustre: Skipped 1 previous similar message [ 9972.255265] Lustre: lustre-MDT0001: new disk, initializing [ 9972.298591] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9972.314874] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9972.320783] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9973.849474] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9977.363724] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9978.866824] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9983.965976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9983.997937] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9984.112069] Lustre: lustre-OST0000: new disk, initializing [ 9984.114621] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9984.147878] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9986.305126] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9986.309251] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9986.342921] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9986.517033] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [ 9990.059397] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9991.613186] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9991.695287] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9993.201710] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9993.292204] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9996.771274] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9996.772535] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9996.773681] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9996.776888] Lustre: Skipped 1 previous similar message [ 9996.783316] LustreError: Skipped 1 previous similar message [10000.177114] LustreError: 434659:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [10000.181702] LustreError: 434659:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [10000.228419] Lustre: server umount lustre-OST0000 complete [10001.889792] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10001.891534] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10001.894362] Lustre: Skipped 1 previous similar message [10001.896831] Lustre: Skipped 1 previous similar message [10007.009112] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10007.011671] Lustre: Skipped 2 previous similar messages [10012.129046] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10012.131248] Lustre: Skipped 1 previous similar message [10016.224138] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10016.259343] LustreError: 434909:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [10016.262237] LustreError: 434909:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [10016.297460] Lustre: server umount lustre-MDT0000 complete [10018.181230] LustreError: 431022:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767324771 with bad export cookie 9635373942872218121 [10018.184713] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10018.185098] LustreError: 431022:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [10018.295712] Lustre: server umount lustre-MDT0001 complete [10019.648778] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10020.293977] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 22:32:52 (1767324772) [10024.032268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10024.190753] LustreError: 435924:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10024.198701] LustreError: 435924:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10024.214107] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10025.565417] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10029.538204] LustreError: 435924:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10030.397648] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10032.006554] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10034.211992] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10035.795281] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10040.189916] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10040.328616] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10040.331512] Lustre: Skipped 1 previous similar message [10042.485815] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10045.516221] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10051.668941] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10051.693505] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10051.751674] Lustre: lustre-OST0001: new disk, initializing [10051.754243] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10053.700533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10053.705136] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10053.723498] LustreError: 439619:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10053.728389] LustreError: 439619:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10053.765238] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10054.755546] LustreError: 439619:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10054.759122] LustreError: 439619:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10055.779893] LustreError: 439619:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10055.783343] LustreError: 439619:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10057.827806] LustreError: 439631:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10057.831078] LustreError: 439631:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10058.006212] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10058.087594] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10059.452598] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9970512b9000.ost_server_uuid 50 [10060.098092] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9970512b9000.ost_server_uuid in FULL state after 0 sec [10061.564082] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9970512b9000.ost_server_uuid 50 [10061.923588] LustreError: 439619:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10061.927479] LustreError: 439619:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10062.108363] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9970512b9000.ost_server_uuid in FULL state after 0 sec [10062.958428] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10074.084211] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10074.088152] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10074.090891] Lustre: Skipped 1 previous similar message [10074.094845] Lustre: Skipped 1 previous similar message [10080.112145] LustreError: 440651:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [10080.114181] LustreError: 440651:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [10080.128791] Lustre: server umount lustre-OST0001 complete [10082.272881] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10082.276389] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10084.324112] LustreError: 439620:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10084.324826] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10084.330039] LustreError: 439620:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10088.031685] Lustre: server umount lustre-OST0000 complete [10092.000842] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10092.003671] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10092.007951] Lustre: Skipped 1 previous similar message [10092.009704] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10092.011406] Lustre: Skipped 6 previous similar messages [10104.289753] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10104.335898] LustreError: 441103:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [10104.338345] LustreError: 441103:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [10104.364104] Lustre: server umount lustre-MDT0000 complete [10104.801127] LustreError: 435942:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10104.807066] LustreError: 435942:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10106.171903] LustreError: 435906:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767324859 with bad export cookie 9635373942872219003 [10106.174324] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10106.176084] LustreError: 435906:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [10106.324709] Lustre: server umount lustre-MDT0001 complete [10111.258372] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [10112.375492] Key type lgssc unregistered [10112.508565] LNet: 442029:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10112.511380] LNetError: 442029:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [10112.517316] LNet: Removed LNI 192.168.203.107@tcp [10112.848895] Key type .llcrypt unregistered [10112.849918] Key type ._llcrypt unregistered [10144.171194] Key type ._llcrypt registered [10144.172816] Key type .llcrypt registered [10144.222186] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [10144.570971] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10144.619861] alg: No test for adler32 (adler32-zlib) [10145.493347] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [10145.580677] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [10147.168146] Key type lgssc registered [10147.550520] Lustre: Echo OBD driver; http://www.lustre.org/ [10150.267367] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10150.280072] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10151.446923] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10152.762348] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10154.670619] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10158.627876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10158.788480] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10160.507644] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10165.563331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10165.693531] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10167.736088] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10168.804415] LustreError: 448420:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10168.811798] LustreError: 448420:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10169.611096] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10173.924501] LustreError: 448420:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10174.279606] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10174.364615] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10176.355388] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10176.430039] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10176.432805] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10178.597429] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10181.150838] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10181.229070] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10258.402392] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10258.404335] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10258.407734] Lustre: Skipped 1 previous similar message [10258.408076] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10258.410447] Lustre: Skipped 1 previous similar message [10263.525570] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10263.527346] Lustre: Skipped 1 previous similar message [10268.643755] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10268.646542] Lustre: Skipped 1 previous similar message [10272.224117] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10272.244134] LustreError: 450673:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [10272.262630] Lustre: server umount lustre-OST0001 complete [10273.763529] LustreError: 448900:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10273.770198] LustreError: 448900:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10276.320910] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10276.323599] LustreError: Skipped 1 previous similar message [10276.324945] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10276.329562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10278.882867] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10278.883103] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10278.884556] LustreError: 450553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10280.045293] LustreError: 450876:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [10280.047352] LustreError: 450876:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [10280.082799] Lustre: server umount lustre-OST0000 complete [10281.953210] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10281.956364] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10281.961524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10281.963878] Lustre: Skipped 2 previous similar messages [10287.744875] LustreError: 451126:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [10287.746635] LustreError: 451126:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [10287.772199] Lustre: server umount lustre-MDT0000 complete [10288.033024] LustreError: 447779:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10288.040254] LustreError: 447779:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10289.243354] LustreError: 451128:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767325042 with bad export cookie 662771057881600906 [10289.247400] LustreError: 451128:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [10289.248340] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10289.363399] Lustre: server umount lustre-MDT0001 complete [10293.855329] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [10294.921548] Key type lgssc unregistered [10295.053716] LNet: 452052:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10295.055891] LNetError: 452052:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [10295.062315] LNet: Removed LNI 192.168.203.107@tcp [10295.362581] Key type .llcrypt unregistered [10295.363508] Key type ._llcrypt unregistered [10334.861689] Key type ._llcrypt registered [10334.862811] Key type .llcrypt registered [10334.908317] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [10335.305460] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10335.324115] alg: No test for adler32 (adler32-zlib) [10336.185500] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [10336.272260] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [10337.856130] Key type lgssc registered [10338.257752] Lustre: Echo OBD driver; http://www.lustre.org/ [10344.279432] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10344.288341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10344.306064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10345.339791] LustreError: 458441:0:(ldlm_resource.c:981:ldlm_resource_complain()) MGC192.168.203.107@tcp: namespace resource [0x65727473756c:0x0:0x0].0x0 (00000000687c285b) refcount nonzero (1) after lock cleanup; forcing cleanup. [10345.417186] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10346.837966] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10346.933633] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10348.801120] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10348.896563] Lustre: 459582:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10352.834589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10352.964659] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10352.967039] Lustre: Skipped 1 previous similar message [10353.829285] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10355.025291] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10355.945068] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10355.946242] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10356.042566] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10360.528409] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10366.243149] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10376.674066] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10376.675539] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10376.678742] Lustre: Skipped 3 previous similar messages [10376.681610] Lustre: Skipped 2 previous similar messages [10381.438613] LustreError: 462608:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [10381.460329] Lustre: server umount lustre-MDT0001 complete [10381.573855] Lustre: server umount lustre-MDT0000 complete [10382.990589] LustreError: 458462:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767325136 with bad export cookie 10201932608220210260 [10382.993897] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10382.995402] LustreError: 458462:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [10383.024526] LustreError: 463090:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [10383.026807] LustreError: 463090:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [10383.049518] Lustre: server umount lustre-OST0000 complete [10383.078942] Lustre: server umount lustre-OST0001 complete [10409.102782] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [10415.121760] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10415.122064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10415.251842] LustreError: 468415:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10415.256866] LustreError: 468415:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10415.274816] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10415.276741] Lustre: Skipped 1 previous similar message [10416.658875] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10416.748304] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10418.626570] Lustre: 469488:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10422.498069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10422.642790] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10422.644753] Lustre: Skipped 1 previous similar message [10423.539546] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10424.688234] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10424.695952] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10424.754079] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10425.773550] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10429.261371] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10430.441684] Lustre: 471943:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10430.444712] Lustre: 471943:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [10440.162137] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10440.163215] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10440.166092] Lustre: Skipped 3 previous similar messages [10440.170361] Lustre: Skipped 4 previous similar messages [10442.690884] LustreError: 472180:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [10442.692810] LustreError: 472180:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10442.724790] Lustre: server umount lustre-MDT0000 complete [10442.815786] Lustre: server umount lustre-MDT0001 complete [10444.153674] LustreError: 468396:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767325197 with bad export cookie 10201932608220211926 [10444.155278] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10444.157297] LustreError: 468396:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [10444.219484] Lustre: server umount lustre-OST0000 complete [10444.256444] Lustre: server umount lustre-OST0001 complete [10446.299284] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10446.831500] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10447.394097] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 22:39:59 (1767325199) [10471.186716] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [10474.023142] loop0: detected capacity change from 0 to 1099511627776 [10474.165465] SGI XFS with ACLs, security attributes, quota, no debug enabled [10474.168283] *** VALIDATE xfs *** [10474.172091] XFS (loop0): Mounting V5 Filesystem [10474.179362] XFS (loop0): Ending clean mount [10475.250728] loop1: detected capacity change from 0 to 18691697672192 [10489.896912] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10500.032049] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [10519.666966] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [10525.981244] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10526.154731] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10526.688540] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10527.542374] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10532.766748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10532.789506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10532.868939] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10532.881992] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10532.917576] Lustre: lustre-MDT0000: new disk, initializing [10532.944571] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10532.947599] Lustre: Skipped 1 previous similar message [10532.954045] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10534.250362] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10540.250463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10540.273972] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10540.309647] Lustre: 486669:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10540.334271] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10540.335793] Lustre: Skipped 1 previous similar message [10540.370181] Lustre: lustre-MDT0001: new disk, initializing [10540.409026] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10540.412920] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10541.649344] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10544.706131] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10545.914405] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10550.123736] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10550.144098] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10550.232733] Lustre: lustre-OST0000: new disk, initializing [10550.235093] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10550.262041] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10550.265211] Lustre: Skipped 1 previous similar message [10551.736621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10551.739277] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10551.773662] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10552.084037] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10555.048124] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10556.358685] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10556.426783] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10557.655896] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10557.722935] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10562.019479] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10562.019966] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10562.020365] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10562.023399] Lustre: Skipped 1 previous similar message [10562.025132] LustreError: Skipped 1 previous similar message [10564.399081] LustreError: 490298:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [10564.401860] LustreError: 490298:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [10564.434535] Lustre: server umount lustre-OST0000 complete [10566.112865] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10566.116345] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10566.120340] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10566.122122] Lustre: Skipped 1 previous similar message [10572.033331] LustreError: 490547:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [10572.036086] LustreError: 490547:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [10572.066802] Lustre: server umount lustre-MDT0000 complete [10572.258055] LustreError: 486682:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10572.262110] LustreError: 486682:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10573.652298] LustreError: 486661:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767325327 with bad export cookie 10201932608220213431 [10573.655401] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10573.657613] LustreError: 486661:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [10573.786262] Lustre: server umount lustre-MDT0001 complete [10576.569443] XFS (loop0): Unmounting Filesystem [10578.128073] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 22:42:10 (1767325330) [10581.565423] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10581.707762] LustreError: 491808:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10581.733579] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10582.999079] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10586.982620] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10588.376045] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10590.109623] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10591.238602] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10594.453487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10596.511316] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10598.398563] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10617.314365] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10617.315480] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10617.316048] LustreError: Skipped 1 previous similar message [10617.316543] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10617.319444] Lustre: Skipped 2 previous similar messages [10617.323650] Lustre: Skipped 3 previous similar messages [10619.375096] LustreError: 494560:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [10619.377710] LustreError: 494560:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [10619.420303] Lustre: server umount lustre-OST0000 complete [10627.223246] Lustre: server umount lustre-MDT0000 complete [10627.553175] LustreError: 494352:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10627.557963] LustreError: 494352:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10628.816744] LustreError: 491789:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767325382 with bad export cookie 10201932608220214306 [10628.818390] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10628.820975] LustreError: 491789:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [10628.934186] Lustre: server umount lustre-MDT0001 complete [10633.533549] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [10634.573072] Key type lgssc unregistered [10634.715548] LNet: 495733:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10634.718385] LNetError: 495733:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [10634.726315] LNet: Removed LNI 192.168.203.107@tcp [10635.042135] Key type .llcrypt unregistered [10635.043424] Key type ._llcrypt unregistered [10665.257267] Key type ._llcrypt registered [10665.258406] Key type .llcrypt registered [10665.296977] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [10665.600800] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10665.611093] alg: No test for adler32 (adler32-zlib) [10666.460027] Lustre: Lustre: Build Version: 2.17.0_RC4_4_g6421352 [10666.542423] LNet: Added LNI 192.168.203.107@tcp [8/256/0/180] [10668.120138] Key type lgssc registered [10668.500372] Lustre: Echo OBD driver; http://www.lustre.org/ [10671.238286] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10671.244325] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10672.417912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10673.715915] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10677.725038] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10677.863098] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10679.281411] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10681.182323] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10682.616411] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10686.194980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10686.309866] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10688.180711] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10689.941335] Lustre: DEBUG MARKER: oleg307-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10705.821880] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.7@tcp (stopping) [10706.914237] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10706.918458] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10706.920432] Lustre: Skipped 1 previous similar message [10710.934393] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.7@tcp (stopping) [10716.055022] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.7@tcp (stopping) [10716.057968] Lustre: Skipped 2 previous similar messages [10718.176108] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10718.212569] LustreError: 503128:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10718.234634] Lustre: server umount lustre-MDT0001 complete [10719.500284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10721.173702] LustreError: 502875:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.7@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10721.178658] LustreError: 502875:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10722.274403] LustreError: 500333:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10722.280876] LustreError: 500333:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10723.255759] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10723.294276] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10723.295943] Lustre: Found index 1 for lustre-MDT0001, updating log [10723.310200] Lustre: 500321:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10723.423482] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10724.721328] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [10728.935539] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10731.495490] LustreError: 504500:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10731.499407] LustreError: 504500:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10731.503607] LustreError: 500324:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10731.523559] LustreError: 504503:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.203.107@tcp: cfg command failed: rc = -17 [10731.528122] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [10731.528122] [11028.381914] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.7@tcp (stopping) [11028.383805] Lustre: Skipped 2 previous similar messages [11031.009917] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11031.010125] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11031.012990] Lustre: Skipped 1 previous similar message [11033.601534] LustreError: 516040:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [11033.604108] LustreError: 516040:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [11033.626835] Lustre: server umount lustre-MDT0001 complete [11034.897327] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11036.129242] LustreError: 500351:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11036.134574] LustreError: 500351:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11038.571842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [11038.597104] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11038.598846] Lustre: Found index 1 for lustre-MDT0001, updating log [11038.614177] LustreError: 500329:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.7@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11038.615827] Lustre: 500321:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11038.738047] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11040.061680] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [11043.818363] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11047.911551] LustreError: 517506:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11047.914373] LustreError: 517506:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11047.917312] LustreError: 500324:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11047.920628] LustreError: 500324:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11047.941040] LustreError: 517509:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [11047.942954] LustreError: 517509:0:(obd_config.c:797:class_detach()) OBD device 8 still set up [11047.944769] LustreError: 517509:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.203.107@tcp: cfg command failed: rc = -17 [11047.948192] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.107@tcp [11047.948192] [11345.818441] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.7@tcp (stopping) [11345.820484] Lustre: Skipped 3 previous similar messages [11345.890030] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11345.891271] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11348.222120] LustreError: 529094:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [11348.224280] LustreError: 529094:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [11348.246976] Lustre: server umount lustre-MDT0001 complete [11349.398876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11350.935217] LustreError: 500329:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.7@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11352.990274] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [11353.018093] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11353.020384] Lustre: Found index 1 for lustre-MDT0001, updating log [11353.036555] Lustre: 500321:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11353.139997] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11354.531390] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_default_debug -1 all [11358.185408] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11359.206358] LustreError: 530410:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11359.208666] LustreError: 530410:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11359.210996] LustreError: 500324:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11359.212980] LustreError: 500324:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11359.229103] LustreError: 530413:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11359.234221] LustreError: 530413:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.203.107@tcp: cfg command failed: rc = -114 [11359.236790] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11359.236790] [11670.497756] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11670.498308] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11670.502076] Lustre: Skipped 2 previous similar messages [11670.503636] Lustre: Skipped 2 previous similar messages [11675.262196] LustreError: 542284:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [11675.264057] LustreError: 542284:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [11675.291629] Lustre: server umount lustre-MDT0000 complete [11675.616888] LustreError: 507567:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11675.621033] LustreError: 507567:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11683.296091] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11683.334785] LustreError: 542262:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [11683.336749] LustreError: 542262:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [11683.382655] Lustre: server umount lustre-MDT0001 complete [11692.001120] Lustre: 499644:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767326429/real 1767326429] req@00000000404f795a x1853175019925888/t0(0) o400->MGC192.168.203.107@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1767326445 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [11692.007665] LustreError: MGC192.168.203.107@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11693.024517] LustreError: 543405:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [11693.026448] LustreError: 543405:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [11693.077443] Lustre: server umount lustre-OST0000 complete [11698.852275] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [11720.403932] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [11727.103570] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11727.247307] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11727.837151] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11728.429336] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11730.550098] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 23:01:23 (1767326483) [11734.725057] Lustre: DEBUG MARKER: conf-sanity test_120: @@@@@@ FAIL: MDT start failed [11744.246485] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing set_hostid [11764.228703] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing load_modules_local [11770.534065] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11770.955477] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11771.576587] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11772.275203] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11772.851433] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11548 sec ========= 23:02:05 (1767326525) [11773.446133] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 23:02:05 (1767326525) === [11774.569409] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 23:02:07 (1767326527) === [11783.454811] Lustre: DEBUG MARKER: oleg307-server.virtnet: executing unload_modules_local [11784.596417] Key type lgssc unregistered [11784.725515] LNet: 560309:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11784.729151] LNetError: 560309:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [11784.737491] LNet: Removed LNI 192.168.203.107@tcp [11785.049550] Key type .llcrypt unregistered [11785.051523] Key type ._llcrypt unregistered