[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 540746653 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003222] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005017] kvm-guest: setup PV IPIs [ 0.007881] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008031] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009016] pid_max: default: 32768 minimum: 301 [ 0.011010] LSM: Security Framework initializing [ 0.012068] Yama: becoming mindful. [ 0.013046] SELinux: Initializing. [ 0.014082] *** VALIDATE selinux *** [ 0.022720] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026812] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028159] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029117] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030151] *** VALIDATE tmpfs *** [ 0.032328] *** VALIDATE proc *** [ 0.033288] *** VALIDATE cgroup *** [ 0.034012] *** VALIDATE cgroup2 *** [ 0.035260] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036163] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038040] Spectre V2 : User space: Vulnerable [ 0.039014] Speculative Store Bypass: Vulnerable [ 0.042535] debug: unmapping init [mem 0xffffffff88059000-0xffffffff88060fff] [ 0.044194] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045734] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046029] ... version: 2 [ 0.047013] ... bit width: 48 [ 0.048014] ... generic registers: 4 [ 0.049016] ... value mask: 0000ffffffffffff [ 0.050022] ... max period: 00007fffffffffff [ 0.051019] ... fixed-purpose events: 3 [ 0.052018] ... event mask: 000000070000000f [ 0.053331] rcu: Hierarchical SRCU implementation. [ 0.055562] smp: Bringing up secondary CPUs ... [ 0.056621] x86: Booting SMP configuration: [ 0.057030] .... node #0, CPUs: #1 #2 #3 [ 0.060387] smp: Brought up 1 node, 4 CPUs [ 0.062013] smpboot: Max logical packages: 1 [ 0.063021] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.144118] node 0 deferred pages initialised in 79ms [ 0.146104] devtmpfs: initialized [ 0.147018] x86/mm: Memory block size: 128MB [ 0.150000] gcov: version magic: 0x41383552 [ 0.153365] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.154154] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.155391] pinctrl core: initialized pinctrl subsystem [ 0.156227] [ 0.156656] ************************************************************* [ 0.157018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.158016] ** ** [ 0.159017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.160016] ** ** [ 0.161014] ** This means that this kernel is built to expose internal ** [ 0.162014] ** IOMMU data structures, which may compromise security on ** [ 0.163014] ** your system. ** [ 0.164015] ** ** [ 0.165013] ** If you see this message and you are not debugging the ** [ 0.166019] ** kernel, report this immediately to your vendor! ** [ 0.167017] ** ** [ 0.168016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.169031] ************************************************************* [ 0.170716] NET: Registered protocol family 16 [ 0.171483] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.172068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.173060] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.174477] cpuidle: using governor menu [ 0.176957] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.179831] PCI: Using configuration type 1 for base access [ 0.182138] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.193134] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.194020] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.196045] cryptd: max_cpu_qlen set to 1000 [ 0.200285] ACPI: Added _OSI(Module Device) [ 0.201012] ACPI: Added _OSI(Processor Device) [ 0.202011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.203018] ACPI: Added _OSI(Processor Aggregator Device) [ 0.207312] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.209602] ACPI: Interpreter enabled [ 0.210061] ACPI: PM: (supports S0 S3 S4 S5) [ 0.211010] ACPI: Using IOAPIC for interrupt routing [ 0.212091] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.213407] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.222824] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.223039] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.224020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.225092] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.227382] acpiphp: Slot [2] registered [ 0.228115] acpiphp: Slot [5] registered [ 0.229094] acpiphp: Slot [6] registered [ 0.230097] acpiphp: Slot [7] registered [ 0.231107] acpiphp: Slot [8] registered [ 0.232079] acpiphp: Slot [9] registered [ 0.233125] acpiphp: Slot [10] registered [ 0.234098] acpiphp: Slot [3] registered [ 0.235095] acpiphp: Slot [4] registered [ 0.236076] acpiphp: Slot [11] registered [ 0.237103] acpiphp: Slot [12] registered [ 0.238094] acpiphp: Slot [13] registered [ 0.239128] acpiphp: Slot [14] registered [ 0.240082] acpiphp: Slot [15] registered [ 0.241091] acpiphp: Slot [16] registered [ 0.242082] acpiphp: Slot [17] registered [ 0.243099] acpiphp: Slot [18] registered [ 0.244087] acpiphp: Slot [19] registered [ 0.245186] acpiphp: Slot [20] registered [ 0.246213] acpiphp: Slot [21] registered [ 0.247170] acpiphp: Slot [22] registered [ 0.248142] acpiphp: Slot [23] registered [ 0.249182] acpiphp: Slot [24] registered [ 0.250142] acpiphp: Slot [25] registered [ 0.251443] acpiphp: Slot [26] registered [ 0.252227] acpiphp: Slot [27] registered [ 0.253149] acpiphp: Slot [28] registered [ 0.254174] acpiphp: Slot [29] registered [ 0.255164] acpiphp: Slot [30] registered [ 0.256169] acpiphp: Slot [31] registered [ 0.257116] PCI host bridge to bus 0000:00 [ 0.258040] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.259038] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.260040] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.261042] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.262038] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.263043] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.264245] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.266090] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.268520] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.274023] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.277763] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.278037] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.279027] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.280038] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.281826] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.282868] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.283086] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.284991] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.287028] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.295032] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.297968] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.301741] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.311015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.321019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.340022] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.351000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.365020] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.374016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.426018] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.437517] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.446019] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.458016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.480020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.491426] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.496015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.504019] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.516016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.524784] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.529015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.533017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.548020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.556000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.565018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.575020] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.603023] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.622964] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.625396] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.627379] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.630473] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.633263] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.637197] iommu: Default domain type: Passthrough [ 0.639438] SCSI subsystem initialized [ 0.640161] ACPI: bus type USB registered [ 0.642116] usbcore: registered new interface driver usbfs [ 0.644077] usbcore: registered new interface driver hub [ 0.646105] usbcore: registered new device driver usb [ 0.647153] pps_core: LinuxPPS API ver. 1 registered [ 0.649022] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.651083] PTP clock support registered [ 0.653159] EDAC MC: Ver: 3.0.0 [ 0.655167] PCI: Using ACPI for IRQ routing [ 0.657768] NetLabel: Initializing [ 0.659018] NetLabel: domain hash size = 128 [ 0.660011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.662106] NetLabel: unlabeled traffic allowed by default [ 0.664088] vgaarb: loaded [ 0.665291] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.667016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.673368] clocksource: Switched to clocksource kvm-clock [ 0.788246] VFS: Disk quotas dquot_6.6.0 [ 0.790125] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.792856] *** VALIDATE ramfs *** [ 0.794389] *** VALIDATE hugetlbfs *** [ 0.796831] pnp: PnP ACPI init [ 0.799551] pnp: PnP ACPI: found 6 devices [ 0.815443] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.819057] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.822401] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.824766] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.827662] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.830467] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.833769] NET: Registered protocol family 2 [ 0.836508] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.842556] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.846717] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.852967] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.856728] TCP: Hash tables configured (established 65536 bind 65536) [ 0.860092] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.863775] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.866987] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.870241] NET: Registered protocol family 1 [ 0.873520] RPC: Registered named UNIX socket transport module. [ 0.876372] RPC: Registered udp transport module. [ 0.878373] RPC: Registered tcp transport module. [ 0.880234] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.882721] NET: Registered protocol family 44 [ 0.884462] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.886986] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.889089] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.891817] PCI: CLS 0 bytes, default 64 [ 0.893615] Unpacking initramfs... [ 2.349930] debug: unmapping init [mem 0xffff962bbcc54000-0xffff962bbffbffff] [ 2.353969] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.356536] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.359721] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.868486] Initialise system trusted keyrings [ 2.870465] Key type blacklist registered [ 2.872955] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.884220] zbud: loaded [ 2.887880] *** VALIDATE nfs *** [ 2.889038] *** VALIDATE nfs4 *** [ 2.891332] pstore: using deflate compression [ 2.894955] Platform Keyring initialized [ 2.999790] NET: Registered protocol family 38 [ 3.001511] Key type asymmetric registered [ 3.002601] Asymmetric key parser 'x509' registered [ 3.004179] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.007186] io scheduler mq-deadline registered [ 3.009229] io scheduler kyber registered [ 3.012901] io scheduler bfq registered [ 3.015314] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.018498] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.021771] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.024716] ACPI: Power Button [PWRF] [ 3.028915] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.034443] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.051698] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.062733] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.080769] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.108517] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.137247] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.143221] Non-volatile memory driver v1.3 [ 3.144754] Linux agpgart interface v0.103 [ 3.181814] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.184629] vda: detected capacity change from 0 to 74698752 [ 3.210292] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.213744] vdb: detected capacity change from 0 to 1073741824 [ 3.231572] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.234895] vdc: detected capacity change from 0 to 2621440000 [ 3.252723] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.255903] vdd: detected capacity change from 0 to 2621440000 [ 3.273616] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.277054] vde: detected capacity change from 0 to 4294967296 [ 3.297381] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.300284] vdf: detected capacity change from 0 to 4294967296 [ 3.314217] libphy: Fixed MDIO Bus: probed [ 3.321926] usbcore: registered new interface driver usbserial_generic [ 3.324229] usbserial: USB Serial support registered for generic [ 3.327090] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.332175] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.333812] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.336427] mousedev: PS/2 mouse device common for all mice [ 3.339273] rtc_cmos 00:05: RTC can wake from S4 [ 3.341755] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.344164] rtc_cmos 00:05: registered as rtc0 [ 3.347208] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.349641] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.351879] intel_pstate: CPU model not supported [ 3.360633] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.361951] hid: raw HID events driver (C) Jiri Kosina [ 3.366523] usbcore: registered new interface driver usbhid [ 3.368884] usbhid: USB HID core driver [ 3.370806] drop_monitor: Initializing network drop monitor service [ 3.373714] Initializing XFRM netlink socket [ 3.376103] NET: Registered protocol family 10 [ 3.380198] Segment Routing with IPv6 [ 3.381807] NET: Registered protocol family 17 [ 3.384489] mpls_gso: MPLS GSO support [ 3.390718] RAS: Correctable Errors collector initialized. [ 3.392439] AVX version of gcm_enc/dec engaged. [ 3.393781] AES CTR mode by8 optimization enabled [ 3.474903] sched_clock: Marking stable (3474866940, 0)->(4518860235, -1043993295) [ 3.479193] registered taskstats version 1 [ 3.481235] Loading compiled-in X.509 certificates [ 3.483467] zswap: loaded using pool lzo/zbud [ 3.510833] Key type big_key registered [ 3.525748] Key type encrypted registered [ 3.527518] ima: No TPM chip found, activating TPM-bypass! [ 3.529989] ima: Allocated hash algorithm: sha1 [ 3.532251] ima: No architecture policies found [ 3.534455] evm: Initialising EVM extended attributes: [ 3.535849] evm: security.selinux [ 3.536846] evm: security.ima [ 3.537889] evm: security.capability [ 3.538896] evm: HMAC attrs: 0x1 [ 3.541248] rtc_cmos 00:05: setting system clock to 2026-08-06 08:48:05 UTC (1786006085) [ 3.549162] debug: unmapping init [mem 0xffffffff89003000-0xffffffff891fffff] [ 3.551332] debug: unmapping init [mem 0xffffffff87d82000-0xffffffff88058fff] [ 3.559102] Write protecting the kernel read-only data: 28672k [ 3.561757] debug: unmapping init [mem 0xffffffff86403000-0xffffffff865fffff] [ 3.564255] debug: unmapping init [mem 0xffffffff86d14000-0xffffffff86dfffff] [ 3.629110] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.635599] systemd[1]: Detected virtualization kvm. [ 3.637049] systemd[1]: Detected architecture x86-64. [ 3.639121] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.669506] systemd[1]: No hostname configured. [ 3.671508] systemd[1]: Set hostname to . [ 3.673773] random: systemd: uninitialized urandom read (16 bytes read) [ 3.676607] systemd[1]: Initializing machine ID from random generator. [ 3.726996] random: ln: uninitialized urandom read (6 bytes read) [ 3.816976] random: systemd: uninitialized urandom read (16 bytes read) [ 3.819508] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.824753] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.830315] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Swap. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Slices. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.465454] device-mapper: uevent: version 1.0.3 [ 4.468960] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] [ 5.113744] random: fast init done Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.275869] virtio_net virtio0 ens2: renamed from eth0 [ 5.330202] scsi host0: ata_piix [ 5.343352] scsi host1: ata_piix [ 5.346151] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.349245] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.979133] dracut-initqueue[594]: RTNETLINK answers: File exists [ 10.051347] random: crng init done [ 10.053151] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.582150] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.679500] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.961214] SELinux: Disabled at runtime. [ 12.027819] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.037886] systemd[1]: Detected virtualization kvm. [ 12.039966] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.584861] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.588808] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.594639] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.597928] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.602491] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.613575] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.617608] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Process Core Dump Socket. Mounting Huge Pages File System... [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Kernel Socket. Starting Remount Root and Kernel File Systems... Activating swap /dev/disk/by-label/SWAP... Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on RPCbind Server Activation Socket. [ 12.727662] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Control Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting udev Coldplug all Devices... [ OK ] Reached target Paths. Mounting POSIX Message Queue File System... [ OK ] Stopped target Initrd File Systems. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 13.013901] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.370593] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.397164] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.484934] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.504119] EDAC sbridge: Ver: 1.1.2 [ 14.811546] Key type dns_resolver registered [ 15.243572] NFS: Registering the id_resolver key type [ 15.245561] Key type id_resolver registered [ 15.248139] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. [ OK ] Started irqbalance daemon. Starting Login Service... Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started Login Service. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg204-server login: [ 79.989629] libcfs: loading out-of-tree module taints kernel. [ 80.067154] Key type ._llcrypt registered [ 80.074063] Key type .llcrypt registered [ 80.204366] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 101.395450] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 103.541423] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 103.557970] alg: No test for adler32 (adler32-zlib) [ 105.301070] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 106.508182] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 108.264171] Key type lgssc registered [ 110.395291] Lustre: Echo OBD driver; http://www.lustre.org/ [ 119.534548] hrtimer: interrupt took 8440521 ns [ 128.275690] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 175.273697] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 191.433683] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 191.492936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 192.843995] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 192.884740] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 192.983971] Lustre: lustre-MDT0000: new disk, initializing [ 193.082784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 193.120369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 199.098535] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 213.428484] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 213.567301] Lustre: 6542:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 213.616842] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 213.621476] Lustre: Skipped 1 previous similar message [ 213.744879] Lustre: lustre-MDT0001: new disk, initializing [ 213.808465] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 213.844415] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 213.866802] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 218.014888] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 223.036653] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 233.692615] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 234.106058] Lustre: lustre-OST0000: new disk, initializing [ 234.112416] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 234.123970] Lustre: 8476:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 234.223641] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 236.914647] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 236.921680] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 236.989684] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 241.268437] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 256.515869] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 256.637377] Lustre: lustre-OST0001: new disk, initializing [ 256.639874] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 256.644706] Lustre: 9551:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 256.732113] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 262.698810] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 262.724255] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 262.812181] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 263.499333] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 275.603748] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 285.383600] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 292.328479] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing check_logdir /tmp/testlogs/ [ 297.803870] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing yml_node [ 303.241060] Lustre: DEBUG MARKER: Client: 2.17.56.50 [ 306.358410] Lustre: DEBUG MARKER: MDS: 2.17.56.50 [ 309.361960] Lustre: DEBUG MARKER: OSS: 2.17.56.50 [ 311.204289] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Aug 6 04:53:10 EDT 2026 [ 331.412765] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 334.006167] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 352.226166] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 352.231247] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 352.261785] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 354.785462] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 354.796614] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 354.814646] Lustre: Skipped 2 previous similar messages [ 358.065234] Lustre: server umount lustre-MDT0000 complete [ 361.077368] LustreError: 6532:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786006443 with bad export cookie 6420371176615387082 [ 361.088323] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 361.093316] LustreError: 6532:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 365.029722] LustreError: 6548:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 365.030930] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 365.031259] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 365.031263] Lustre: Skipped 3 previous similar messages [ 365.063461] LustreError: 6548:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 365.100571] Lustre: Skipped 1 previous similar message [ 366.051758] LustreError: 6549:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 366.097323] LustreError: 6549:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 367.575932] Lustre: server umount lustre-MDT0001 complete [ 371.914942] Lustre: server umount lustre-OST0000 complete [ 377.130210] Lustre: server umount lustre-OST0001 complete [ 385.518274] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 397.100527] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 439.178864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 439.588729] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 439.642538] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 439.764468] Lustre: lustre-MDT0000: new disk, initializing [ 439.944681] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 439.970496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 445.276500] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 457.419253] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 457.625308] Lustre: 15111:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 457.770042] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 457.774673] Lustre: Skipped 1 previous similar message [ 457.940389] Lustre: lustre-MDT0001: new disk, initializing [ 458.139126] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 458.220036] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 458.247443] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 464.051994] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 474.586718] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 479.946304] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 487.081457] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 487.530478] Lustre: lustre-OST0000: new disk, initializing [ 487.541884] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 487.550621] Lustre: 17078:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 487.709498] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 489.554665] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 489.568078] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 489.781295] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 495.650586] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 506.319517] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 512.062327] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 512.450640] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 518.316616] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 518.601501] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 525.797824] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 525.804101] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 525.809935] LustreError: Skipped 1 previous similar message [ 525.822785] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 525.839560] Lustre: Skipped 1 previous similar message [ 525.842312] Lustre: Skipped 3 previous similar messages [ 527.312875] Lustre: server umount lustre-OST0000 complete [ 535.011957] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 535.024878] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 535.050846] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 535.056763] Lustre: Skipped 1 previous similar message [ 537.706419] Lustre: server umount lustre-MDT0000 complete [ 541.169354] LustreError: 15121:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 541.190802] LustreError: 15121:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 541.870874] LustreError: 15101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786006623 with bad export cookie 6420371176615388615 [ 541.900257] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 541.902544] LustreError: 15101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 542.413184] Lustre: server umount lustre-MDT0001 complete [ 548.008469] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:57:07 (1786006627) [ 555.877091] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 556.675265] LustreError: 19342:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 556.833838] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 562.157621] LustreError: 19343:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 562.831626] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 571.362797] LustreError: 19342:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 571.409509] LustreError: 19342:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 573.436122] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 578.988599] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 589.944044] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 598.769211] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 607.751187] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 617.145254] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 626.507309] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 630.718085] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 640.483700] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 640.498805] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 640.505403] Lustre: Skipped 1 previous similar message [ 640.510549] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 640.514960] Lustre: Skipped 2 previous similar messages [ 644.624548] Lustre: server umount lustre-MDT0000 complete [ 647.139301] LustreError: 19343:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 647.169617] LustreError: 19343:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 648.747066] LustreError: 19323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786006730 with bad export cookie 6420371176615389497 [ 648.761271] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 648.768284] LustreError: 19323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 649.247073] Lustre: server umount lustre-MDT0001 complete [ 664.274799] Lustre: server umount lustre-OST0000 complete [ 688.255354] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 701.132465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 701.602081] LustreError: 23728:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 701.718166] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 701.733972] Lustre: Skipped 2 previous similar messages [ 707.314252] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 719.048536] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 725.932857] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 729.856181] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 737.549074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 747.116166] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 759.893751] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 760.039115] Lustre: lustre-OST0001: new disk, initializing [ 760.041220] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 760.053300] Lustre: 26044:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 761.840737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 761.866434] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 762.076091] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 769.191366] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 782.576066] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 787.150255] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 806.881745] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 806.890184] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 806.902611] Lustre: Skipped 2 previous similar messages [ 806.907599] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 806.919527] Lustre: Skipped 3 previous similar messages [ 809.836603] Lustre: server umount lustre-MDT0000 complete [ 813.552386] LustreError: 26575:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 813.595763] LustreError: 26575:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 814.116967] LustreError: 26043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786006896 with bad export cookie 6420371176615390344 [ 814.124122] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 814.133053] LustreError: 26043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 814.636902] Lustre: server umount lustre-MDT0001 complete [ 829.177121] Lustre: server umount lustre-OST0000 complete [ 844.034825] Lustre: server umount lustre-OST0001 complete [ 862.094271] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 873.776705] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 874.302760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 874.312863] Lustre: Skipped 3 previous similar messages [ 879.698701] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 889.467912] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 897.166567] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 901.286788] Lustre: 30228:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 909.625026] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 916.742526] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 928.334083] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 937.808912] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 947.172474] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 952.670704] Lustre: 32109:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 962.026220] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 962.044735] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 962.056251] Lustre: Skipped 3 previous similar messages [ 962.061189] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 962.063706] Lustre: Skipped 4 previous similar messages [ 967.317157] Lustre: server umount lustre-MDT0000 complete [ 969.709470] LustreError: 30596:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 969.739763] LustreError: 30596:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 971.598865] LustreError: 29059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786007053 with bad export cookie 6420371176615391639 [ 971.610245] LustreError: 29059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 971.616847] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 972.026105] Lustre: server umount lustre-MDT0001 complete [ 986.812374] Lustre: server umount lustre-OST0000 complete [ 990.177648] Lustre: 3674:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786007056/real 1786007056] req@ffff962b03642d80 x1872763224906240/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786007072 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 990.817630] Lustre: server umount lustre-OST0001 complete [ 1000.345686] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 05:04:40 (1786007080) [ 1034.201436] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1047.156822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1053.803274] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1064.262558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1070.684331] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1074.410693] Lustre: 36065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1082.860713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1092.648925] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1102.657665] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1110.565712] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1119.328575] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1124.621926] Lustre: 37948:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1134.065128] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1134.086710] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1134.087968] Lustre: Skipped 6 previous similar messages [ 1134.108789] Lustre: Skipped 6 previous similar messages [ 1139.408780] Lustre: server umount lustre-MDT0000 complete [ 1144.379624] LustreError: 36067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786007226 with bad export cookie 6420371176615392745 [ 1144.383180] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1144.392426] LustreError: 36067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1144.736316] Lustre: server umount lustre-MDT0001 complete [ 1159.933119] Lustre: server umount lustre-OST0000 complete [ 1175.650580] Lustre: server umount lustre-OST0001 complete [ 1185.809473] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 05:07:45 (1786007265) [ 1204.682733] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1217.271725] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1217.737071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1217.743254] Lustre: Skipped 7 previous similar messages [ 1222.785562] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1227.236268] LustreError: 40091:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1227.291563] LustreError: 40091:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 19 previous similar messages [ 1233.348625] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1240.668708] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1244.570506] Lustre: 41239:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1253.388899] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1262.163411] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1273.329488] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1281.922273] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1291.348743] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1297.016645] Lustre: 43120:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1298.460906] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1298.470250] Lustre: Skipped 1 previous similar message [ 1304.545644] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1304.570375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1304.575278] Lustre: Skipped 3 previous similar messages [ 1304.598382] Lustre: Skipped 4 previous similar messages [ 1308.228088] Lustre: server umount lustre-MDT0000 complete [ 1312.137908] LustreError: 40070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786007394 with bad export cookie 6420371176615393851 [ 1312.149372] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1312.152946] LustreError: 40070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1318.909680] Lustre: server umount lustre-MDT0001 complete [ 1329.360560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1337.313385] LustreError: 3673:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff962b026fdf80 x1872763225019904/t0(0) o250->MGC192.168.202.104@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1342.313108] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1347.080135] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1353.327551] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1359.360306] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1359.373285] Lustre: Skipped 1 previous similar message [ 1360.575689] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1371.540649] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1373.370939] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1380.147916] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1382.161921] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1389.538463] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1395.566833] Lustre: server umount lustre-MDT0000 complete [ 1399.979194] LustreError: 43533:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786007481 with bad export cookie 6420371176615394950 [ 1400.005879] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1400.007077] LustreError: 43533:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1400.500076] Lustre: server umount lustre-MDT0001 complete [ 1408.244995] Lustre: server umount lustre-OST0000 complete [ 1413.454882] Lustre: server umount lustre-OST0001 complete [ 1425.229796] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 05:11:44 (1786007504) [ 1446.788768] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1459.851200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1465.917811] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1477.441671] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1484.264740] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1487.981067] Lustre: 48885:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1496.694641] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1505.329561] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1515.884886] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1523.407420] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1533.183323] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1539.998940] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1562.598201] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1562.619465] Lustre: Skipped 18 previous similar messages [ 1563.905436] Lustre: server umount lustre-MDT0000 complete [ 1568.422877] LustreError: 47714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786007650 with bad export cookie 6420371176615395776 [ 1568.428777] LustreError: 47714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1568.432491] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1568.884022] Lustre: server umount lustre-MDT0001 complete [ 1582.436433] Lustre: server umount lustre-OST0000 complete [ 1597.200905] Lustre: server umount lustre-OST0001 complete [ 1606.818453] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 05:14:46 (1786007686) [ 1609.809089] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1612.153379] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 05:14:51 (1786007691) [ 1625.193738] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 1634.674149] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1660.425270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1660.712937] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1660.788800] Lustre: lustre-MDT0000: new disk, initializing [ 1660.894241] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1666.664929] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1676.478952] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1684.095751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1684.529647] Lustre: lustre-OST0000: new disk, initializing [ 1684.537639] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1684.544870] Lustre: Skipped 1 previous similar message [ 1684.556944] Lustre: 54987:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1686.489906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1686.500044] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1686.557611] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1691.545257] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1702.198744] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1705.007485] Lustre: DEBUG MARKER: create test files [ 1742.840177] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1742.858053] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1742.877631] Lustre: Skipped 13 previous similar messages [ 1747.473202] Lustre: server umount lustre-OST0000 complete [ 1752.145882] Lustre: server umount lustre-MDT0000 complete [ 1765.546890] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 1769.386583] Key type lgssc unregistered [ 1769.776693] LNet: 56621:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1769.781559] LNetError: 56621:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1769.794312] LNet: Removed LNI 192.168.202.104@tcp [ 1770.851233] Key type .llcrypt unregistered [ 1770.856821] Key type ._llcrypt unregistered [ 1804.985755] Key type ._llcrypt registered [ 1804.993273] Key type .llcrypt registered [ 1805.154602] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1807.410948] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1807.468795] alg: No test for adler32 (adler32-zlib) [ 1808.640905] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 1809.014567] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 1810.768178] Key type lgssc registered [ 1812.351270] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1821.776464] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1821.808501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1823.314298] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1829.422677] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1839.970892] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1848.022219] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1848.690236] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1849.710680] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1849.807051] Lustre: 59362:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1849.816750] Lustre: 59362:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1849.827452] Lustre: 59362:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1849.840684] Lustre: 59362:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1849.851532] Lustre: 59362:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1849.859643] Lustre: 59362:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1855.680151] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1863.744564] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1869.228974] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1875.694967] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1885.665515] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1885.679313] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1885.704617] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1886.111432] Lustre: server umount lustre-OST0000 complete [ 1890.390412] Lustre: server umount lustre-MDT0000 complete [ 1902.095705] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 1905.282550] Key type lgssc unregistered [ 1905.652598] LNet: 60972:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1905.668627] LNetError: 60972:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1905.692077] LNet: Removed LNI 192.168.202.104@tcp [ 1906.714300] Key type .llcrypt unregistered [ 1906.722720] Key type ._llcrypt unregistered [ 1942.962654] Key type ._llcrypt registered [ 1942.964448] Key type .llcrypt registered [ 1943.146197] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 1945.457340] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1945.482347] alg: No test for adler32 (adler32-zlib) [ 1946.787323] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 1947.157657] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 1948.872431] Key type lgssc registered [ 1950.499057] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1961.066733] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1961.106442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1962.789787] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1969.610576] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 1979.566585] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1988.717546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1989.150076] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1996.821761] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2005.684385] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2007.553540] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2007.702768] Lustre: 63817:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 2007.708120] Lustre: 63817:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 2007.717123] Lustre: 63817:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2007.730637] Lustre: 63817:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2007.743186] Lustre: 63817:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2007.753029] Lustre: 63817:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2018.344826] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2028.003120] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2028.025200] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2028.068452] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2033.294763] Lustre: server umount lustre-OST0000 complete [ 2038.325306] Lustre: server umount lustre-MDT0000 complete [ 2050.459817] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 2053.258905] Key type lgssc unregistered [ 2053.518574] LNet: 65401:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2053.525347] LNetError: 65401:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2054.566663] LNet: Removed LNI 192.168.202.104@tcp [ 2055.586201] Key type .llcrypt unregistered [ 2055.588454] Key type ._llcrypt unregistered [ 2074.913303] Key type ._llcrypt registered [ 2074.922286] Key type .llcrypt registered [ 2075.100503] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 2092.033878] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 2093.625084] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2093.721558] alg: No test for adler32 (adler32-zlib) [ 2094.903934] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 2095.334359] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 2097.041959] Key type lgssc registered [ 2098.603585] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2138.245306] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2138.308887] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2139.699952] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2139.771923] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2139.908734] Lustre: lustre-MDT0000: new disk, initializing [ 2140.054821] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2140.077658] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2145.003217] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2156.830590] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2156.941333] Lustre: 68555:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2156.989169] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2156.992730] Lustre: Skipped 1 previous similar message [ 2157.061238] Lustre: lustre-MDT0001: new disk, initializing [ 2157.174074] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2157.242332] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2157.265352] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2163.042726] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2174.843275] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2182.179047] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2190.123419] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2190.409583] Lustre: lustre-OST0000: new disk, initializing [ 2190.412773] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2190.417944] Lustre: 70529:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2190.498739] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2197.634713] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2198.108281] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2198.127057] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2198.341434] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2208.475890] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2213.733320] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2214.043110] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2219.254728] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2219.412419] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2224.104617] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2224.110808] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2224.132279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2227.646117] Lustre: server umount lustre-OST0000 complete [ 2233.834288] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2233.849899] LustreError: Skipped 1 previous similar message [ 2233.851626] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2233.878286] Lustre: Skipped 1 previous similar message [ 2233.891574] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2233.899560] Lustre: Skipped 1 previous similar message [ 2238.328926] Lustre: server umount lustre-MDT0000 complete [ 2242.404467] LustreError: 69352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786008324 with bad export cookie 9771154575213750445 [ 2242.409497] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2242.412125] LustreError: 69352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2243.012440] Lustre: server umount lustre-MDT0001 complete [ 2252.324065] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 05:25:31 (1786008331) [ 2263.291744] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 2309.223217] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 2320.111759] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 2360.842841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2361.196697] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2361.221458] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2361.295988] Lustre: lustre-MDT0000: new disk, initializing [ 2361.385153] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2361.409711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2366.166318] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2377.265324] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2377.425472] Lustre: 76552:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2377.491754] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2377.497192] Lustre: Skipped 1 previous similar message [ 2377.591254] Lustre: lustre-MDT0001: new disk, initializing [ 2377.708867] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2377.768990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2377.776447] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2382.840055] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2393.822822] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2400.093582] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2406.475812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2406.675402] Lustre: lustre-OST0000: new disk, initializing [ 2406.678537] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2406.686719] Lustre: 78522:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2406.773247] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2413.659961] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2416.243138] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2416.258626] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2416.348886] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2425.203770] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2430.137616] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2430.388347] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2435.036309] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2435.251827] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2442.212367] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2442.216293] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2442.219297] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2442.219307] Lustre: Skipped 2 previous similar messages [ 2442.234698] Lustre: Skipped 2 previous similar messages [ 2442.259690] LustreError: Skipped 1 previous similar message [ 2443.863922] Lustre: server umount lustre-OST0000 complete [ 2449.379414] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2449.390446] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2449.401127] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2449.403803] Lustre: Skipped 1 previous similar message [ 2454.547861] Lustre: server umount lustre-MDT0000 complete [ 2457.572214] LustreError: 76561:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2457.587337] LustreError: 76561:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2458.769749] LustreError: 78521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786008540 with bad export cookie 9771154575213751327 [ 2458.782518] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2458.786961] LustreError: 78521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2459.076760] Lustre: server umount lustre-MDT0001 complete [ 2469.786717] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 05:29:09 (1786008549) [ 2478.443965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2479.100376] LustreError: 80934:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2479.197662] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2484.193798] LustreError: 80935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2485.229858] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2489.319854] LustreError: 80934:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2494.440379] LustreError: 80935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2496.766685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2502.877233] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2513.825746] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2522.103845] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2531.121235] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2531.951539] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2531.955539] Lustre: Skipped 1 previous similar message [ 2538.607727] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2549.310648] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2553.733816] Lustre: 83609:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 2dfe627c-c347-4fc8-931d-b945349ed091 at adminstrative request [ 2554.860378] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2554.901403] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2564.068961] Lustre: MGS: Client 2dfe627c-c347-4fc8-931d-b945349ed091 (at 0@lo) reconnecting [ 2564.090153] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x879a1d50fd0c6f91 to 0x879a1d50fd0c7231 [ 2564.119328] Lustre: MGC192.168.202.104@tcp: Connection restored to 0@lo (at 0@lo) [ 2594.092493] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2594.469481] Lustre: lustre-OST0001: new disk, initializing [ 2594.475754] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2594.489749] Lustre: 84003:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2603.030702] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2603.130925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2603.161849] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2603.374438] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2615.405825] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2618.852565] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2618.869853] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2618.903462] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2618.910149] Lustre: Skipped 2 previous similar messages [ 2618.927669] Lustre: Skipped 3 previous similar messages [ 2624.738648] Lustre: server umount lustre-OST0001 complete [ 2629.107198] LustreError: 82759:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2631.650916] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2631.664809] LustreError: Skipped 1 previous similar message [ 2631.670967] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2631.684861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2631.687367] Lustre: Skipped 2 previous similar messages [ 2635.912817] Lustre: server umount lustre-OST0000 complete [ 2649.569675] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2649.582884] Lustre: Skipped 6 previous similar messages [ 2655.202595] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2655.374116] Lustre: server umount lustre-MDT0000 complete [ 2659.814843] LustreError: 80935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2659.839753] LustreError: 80935:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2660.676047] LustreError: 80916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786008742 with bad export cookie 9771154575213752881 [ 2660.684674] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2660.688342] LustreError: 80916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2661.171474] Lustre: server umount lustre-MDT0001 complete [ 2672.542713] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 05:32:31 (1786008751) [ 2674.924635] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2676.767517] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 05:32:36 (1786008756) [ 2678.442815] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2680.509707] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 05:32:39 (1786008759) [ 2682.436604] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2684.480886] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 05:32:43 (1786008763) [ 2685.945356] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2687.859597] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 05:32:47 (1786008767) [ 2689.587433] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2696.385130] loop: module loaded [ 2696.407913] loop0: detected capacity change from 0 to 4294967296 [ 2696.479526] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2698.577603] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2703.315501] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2708.176377] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2740.018033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2740.119666] LustreError: 88133:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2740.125849] LustreError: 88133:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2740.131602] LustreError: 88133:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2740.143359] LustreError: 88133:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2762.716934] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 2773.203853] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 2818.068773] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2818.432552] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2818.467662] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2818.550408] Lustre: lustre-MDT0000: new disk, initializing [ 2818.621629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2818.633396] Lustre: Skipped 1 previous similar message [ 2818.650320] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2824.845401] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2836.995494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2837.173632] Lustre: 91000:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2837.625576] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2842.859241] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2854.783567] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2861.871950] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2869.739302] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2870.148943] Lustre: 92970:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2878.061122] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2879.559331] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2879.695218] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2888.516203] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2893.759428] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2894.049748] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2899.914532] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2900.211871] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2905.578664] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2905.581681] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2905.594047] LustreError: Skipped 2 previous similar messages [ 2905.601234] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2905.601243] Lustre: Skipped 2 previous similar messages [ 2905.631082] Lustre: Skipped 4 previous similar messages [ 2908.545350] Lustre: server umount lustre-OST0000 complete [ 2919.456643] Lustre: server umount lustre-MDT0000 complete [ 2920.941958] LustreError: 91012:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2920.968952] LustreError: 91012:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2923.721940] LustreError: 90993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786009005 with bad export cookie 9771154575213753497 [ 2923.725629] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2923.739571] LustreError: 90993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2924.174830] Lustre: server umount lustre-MDT0001 complete [ 2935.178854] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 05:36:54 (1786009014) [ 2943.538476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2949.629440] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2958.768948] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2959.095499] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2959.109415] Lustre: Skipped 3 previous similar messages [ 2964.395716] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 2972.796454] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2979.730834] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2987.451275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2996.196416] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3007.198053] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3015.026893] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3015.569802] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3022.664233] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3023.133566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3032.869103] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3033.096728] Lustre: lustre-OST0001: new disk, initializing [ 3033.101328] Lustre: Skipped 2 previous similar messages [ 3033.106576] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3033.111886] Lustre: Skipped 3 previous similar messages [ 3033.118397] Lustre: 98766:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3038.848543] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3038.871784] Lustre: Skipped 2 previous similar messages [ 3038.885872] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3038.994457] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3040.542686] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3052.836973] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3060.666980] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3060.932943] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3065.299154] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3065.425179] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3085.587035] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3112.142656] Lustre: Failing over lustre-MDT0000 [ 3112.591733] Lustre: server umount lustre-MDT0000 complete [ 3112.930598] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3112.952837] LustreError: Skipped 1 previous similar message [ 3112.962200] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3113.001745] Lustre: Skipped 2 previous similar messages [ 3113.019668] LustreError: 95392:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3113.056404] LustreError: 95392:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3131.361349] Lustre: 66780:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786009197/real 1786009197] req@000000008faa601d x1872765310497280/t0(0) o400->MGC192.168.202.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786009213 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3131.387470] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3139.454872] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3139.459247] LDISKFS-fs (dm-0): recovery complete [ 3139.474400] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3141.601498] LustreError: 66779:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000dec1aef8 x1872765310504064/t0(0) o250->MGC192.168.202.104@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3141.976821] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3145.216595] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3145.218028] Lustre: 95388:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3147.244429] Lustre: 96120:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3147.258850] Lustre: 96120:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 3147.288981] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 3147.459816] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3147.784283] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3147.804027] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3148.138232] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3148.144549] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 3148.465569] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3148.480851] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 3148.490966] Lustre: 101614:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3148.507811] Lustre: 101614:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 3149.181098] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3149.198467] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 3149.520970] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3149.527198] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 3150.786496] Lustre: 101614:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3150.825431] Lustre: 101614:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 3151.564176] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3151.572399] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 3151.888133] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3151.891302] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 3155.057071] Lustre: 101614:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3155.079203] Lustre: 101614:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 3155.722974] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3155.734354] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 3156.056260] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3156.076090] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 3163.106313] Lustre: 101614:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3163.122372] Lustre: 101614:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 3163.774562] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3163.780020] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 3164.112459] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3164.123760] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 3179.313448] Lustre: 101614:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3179.324125] Lustre: 101614:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 3179.977373] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3180.003985] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 3180.337157] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3180.352478] LustreError: 101614:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 3205.192452] Lustre: 101614:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3205.204313] Lustre: 101614:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3205.211395] Lustre: 101614:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client d0b94ced-bb72-4a2b-bfa9-a5675d8dc0ac@192.168.202.4@tcp [ 3205.244945] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3205.280281] Lustre: 101614:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000d6ed3e95 x1872765291403136/t0(8589934777) o101->d0b94ced-bb72-4a2b-bfa9-a5675d8dc0ac@192.168.202.4@tcp:172/0 lens 592/0 e 0 to 0 dl 1786009297 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3205.318141] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3205.331489] Lustre: lustre-MDT0000: Denying connection for new client d0b94ced-bb72-4a2b-bfa9-a5675d8dc0ac (at 192.168.202.4@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3205.344561] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3205.393898] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3205.397356] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3205.398911] Lustre: Skipped 2 previous similar messages [ 3205.436592] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3205.441138] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3221.474341] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3221.487548] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3221.507806] Lustre: Skipped 3 previous similar messages [ 3221.511795] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3221.520650] Lustre: Skipped 4 previous similar messages [ 3227.347855] Lustre: server umount lustre-OST0000 complete [ 3238.249996] Lustre: server umount lustre-OST0001 complete [ 3257.312175] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3257.698639] Lustre: server umount lustre-MDT0000 complete [ 3259.874737] LustreError: 95387:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3259.880867] LustreError: 95387:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 43 previous similar messages [ 3262.395187] LustreError: 95374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786009344 with bad export cookie 9771154575213804716 [ 3262.434759] LustreError: 95374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3262.438433] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3262.888268] Lustre: server umount lustre-MDT0001 complete [ 3271.888802] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 05:42:31 (1786009351) [ 3280.090501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3280.100445] Lustre: *** cfs_fail_loc=197, val=0*** [ 3295.585089] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3300.706526] LustreError: 103589:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.104@tcp: failed processing log, type 4: rc = -110 [ 3305.923877] Lustre: *** cfs_fail_loc=197, val=0*** [ 3305.925326] Lustre: Skipped 65 previous similar messages [ 3326.561581] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3326.582052] Lustre: Skipped 3 previous similar messages [ 3337.137278] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3345.670738] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3348.510386] Lustre: server umount lustre-OST0000 complete [ 3360.550756] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 05:43:59 (1786009439) [ 3385.576442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3391.915104] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3404.553264] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3409.233141] LustreError: 105243:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3409.256604] LustreError: 105243:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3409.271433] LustreError: 105243:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 24, retries 0, failed: rc = -5 [ 3409.963458] Lustre: server umount lustre-MDT0000 complete [ 3446.498255] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 3458.969151] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 3503.751336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3504.213457] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3504.275919] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3504.433901] Lustre: lustre-MDT0000: new disk, initializing [ 3504.646488] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3511.763720] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3522.549380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3522.690647] Lustre: 109422:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3522.958764] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3527.702429] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3537.671671] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3543.982776] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3550.996630] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3551.258664] Lustre: 111390:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3558.238656] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3558.722565] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3567.928337] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3572.936932] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3573.137869] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3577.513250] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3577.757189] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3584.487995] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3584.488493] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3584.490841] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3584.490849] Lustre: Skipped 12 previous similar messages [ 3584.508235] Lustre: Skipped 6 previous similar messages [ 3584.510025] LustreError: Skipped 3 previous similar messages [ 3594.209119] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3594.487167] Lustre: server umount lustre-OST0000 complete [ 3605.865388] Lustre: server umount lustre-MDT0000 complete [ 3610.087494] LustreError: 112413:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3610.137089] LustreError: 112413:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 3611.572195] LustreError: 109413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786009693 with bad export cookie 9771154575213815370 [ 3611.578940] LustreError: 109413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3611.581589] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3611.855311] Lustre: server umount lustre-MDT0001 complete [ 3621.103095] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:48:21 (1786009701) [ 3634.108836] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 3638.166632] Key type lgssc unregistered [ 3638.464843] LNet: 113883:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3638.474076] LNetError: 113883:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3638.487513] LNet: Removed LNI 192.168.202.104@tcp [ 3639.463231] Key type .llcrypt unregistered [ 3639.465113] Key type ._llcrypt unregistered [ 3658.388230] Key type ._llcrypt registered [ 3658.390532] Key type .llcrypt registered [ 3658.485992] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 3674.133925] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 3675.285776] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3675.315546] alg: No test for adler32 (adler32-zlib) [ 3676.503374] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 3676.795863] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 3678.472744] Key type lgssc registered [ 3679.892674] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3731.697911] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3731.718357] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3733.052527] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3733.077963] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3733.164777] Lustre: lustre-MDT0000: new disk, initializing [ 3733.233288] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3733.244749] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3737.739705] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3749.443109] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3749.678841] Lustre: lustre-OST0000: new disk, initializing [ 3749.681331] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3749.684171] Lustre: Skipped 1 previous similar message [ 3749.687477] Lustre: 118061:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3749.727738] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3755.069876] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3757.639415] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3757.649699] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3757.686617] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3766.009118] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3766.198863] Lustre: lustre-OST0001: new disk, initializing [ 3766.203878] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3766.214161] Lustre: 118924:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3766.315826] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3767.303307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3767.314306] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3767.349955] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3771.151187] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 3782.629904] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3782.643041] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3782.655282] Lustre: Skipped 1 previous similar message [ 3782.685977] Lustre: Skipped 1 previous similar message [ 3787.337539] Lustre: server umount lustre-MDT0000 complete [ 3795.741991] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3803.105361] Lustre: 115263:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786009869/real 1786009869] req@0000000072ae7251 x1872766968029824/t0(0) o400->MGC192.168.202.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786009885 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3803.165458] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3804.430984] Lustre: server umount lustre-OST0000 complete [ 3810.331234] Lustre: server umount lustre-OST0001 complete [ 3821.681583] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 05:51:40 (1786009900) [ 3849.784162] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 05:52:08 (1786009928) [ 3862.795919] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 3892.329369] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:52:51 (1786009971) [ 3935.738227] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 3945.394314] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 4005.100183] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4005.465544] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4005.533613] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4005.668586] Lustre: lustre-MDT0000: new disk, initializing [ 4005.801117] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4005.864161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4011.136404] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4021.771311] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4021.996450] Lustre: 126681:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4022.115658] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4022.119686] Lustre: Skipped 1 previous similar message [ 4022.202203] Lustre: lustre-MDT0001: new disk, initializing [ 4022.292116] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4022.333709] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4022.345549] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4027.750901] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4038.293758] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4044.880564] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4053.743452] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4054.221446] Lustre: lustre-OST0000: new disk, initializing [ 4054.225567] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4054.231225] Lustre: 128652:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4054.325827] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4059.698401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4059.703603] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4059.784082] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4061.308677] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4071.559216] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4077.719733] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4077.909387] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4083.957590] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4084.350418] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4090.340064] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4090.340410] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4090.354285] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4090.366278] Lustre: Skipped 1 previous similar message [ 4090.385066] Lustre: Skipped 1 previous similar message [ 4094.334167] Lustre: server umount lustre-OST0000 complete [ 4100.585313] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4100.601052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4105.705802] Lustre: server umount lustre-MDT0000 complete [ 4110.820914] LustreError: 126692:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4110.870707] LustreError: 126692:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 4111.583931] LustreError: 126672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786010193 with bad export cookie 6915435218858300130 [ 4111.600409] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4112.145584] Lustre: server umount lustre-MDT0001 complete [ 4124.372945] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:56:43 (1786010203) [ 4132.992754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4133.514404] LustreError: 131066:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4133.600676] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4138.616365] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4138.978619] LustreError: 131067:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4143.075436] LustreError: 131066:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4148.214766] LustreError: 131067:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4148.752566] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4153.992447] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4163.071120] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4171.051041] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4178.810157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4178.939690] LustreError: 132890:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4179.498842] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4179.519768] Lustre: Skipped 1 previous similar message [ 4187.976367] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4197.251859] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4204.699269] Lustre: *** cfs_fail_loc=159, val=0*** [ 4220.029239] Lustre: lustre-MDT0000: Client d23d1bbe-2496-4d56-bbca-49f322dd77de (at 192.168.202.4@tcp) reconnecting [ 4222.961831] Lustre: *** cfs_fail_loc=159, val=0*** [ 4222.963706] Lustre: Skipped 5 previous similar messages [ 4239.419807] Lustre: lustre-MDT0000: Client d23d1bbe-2496-4d56-bbca-49f322dd77de (at 192.168.202.4@tcp) reconnecting [ 4251.107141] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4251.110825] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4251.125077] Lustre: Skipped 1 previous similar message [ 4251.136514] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4251.158935] Lustre: Skipped 2 previous similar messages [ 4256.242979] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4256.260341] Lustre: Skipped 1 previous similar message [ 4260.832206] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4260.983513] Lustre: server umount lustre-OST0000 complete [ 4266.979393] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4266.985540] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4266.993344] Lustre: Skipped 1 previous similar message [ 4266.999296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4281.312510] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4281.595956] Lustre: server umount lustre-MDT0000 complete [ 4281.839515] LustreError: 131061:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4281.872890] LustreError: 131061:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4286.593679] LustreError: 133668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786010368 with bad export cookie 6915435218858301005 [ 4286.600371] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4286.610779] LustreError: 133668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 4287.173207] Lustre: server umount lustre-MDT0001 complete [ 4302.821626] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 4307.455994] Key type lgssc unregistered [ 4307.811842] LNet: 134848:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4307.822359] LNetError: 134848:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4307.845747] LNet: Removed LNI 192.168.202.104@tcp [ 4309.050224] Key type .llcrypt unregistered [ 4309.052403] Key type ._llcrypt unregistered [ 4337.121399] Key type ._llcrypt registered [ 4337.123084] Key type .llcrypt registered [ 4337.320987] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 4338.975289] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4339.145408] alg: No test for adler32 (adler32-zlib) [ 4340.332420] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 4340.648692] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 4342.361233] Key type lgssc registered [ 4343.751584] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4353.418636] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4353.438550] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4355.126831] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4361.387247] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4365.283488] LustreError: 136263:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4365.305474] LustreError: 136263:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4370.406629] LustreError: 136262:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4372.742590] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4373.151631] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4378.703975] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4388.434675] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4395.173712] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4404.385204] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4404.779562] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4412.273257] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4421.011084] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4425.342130] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4434.419585] Lustre: *** cfs_fail_loc=159, val=0*** [ 4450.895765] Lustre: lustre-MDT0000: Client 10197e11-b463-4200-9e66-17d581854183 (at 192.168.202.4@tcp) reconnecting [ 4453.852779] Lustre: *** cfs_fail_loc=159, val=0*** [ 4469.333325] Lustre: lustre-MDT0001: Client 10197e11-b463-4200-9e66-17d581854183 (at 192.168.202.4@tcp) reconnecting [ 4471.471470] Lustre: *** cfs_fail_loc=159, val=0*** [ 4471.488418] Lustre: Skipped 3 previous similar messages [ 4487.768186] Lustre: lustre-MDT0001: Client 10197e11-b463-4200-9e66-17d581854183 (at 192.168.202.4@tcp) reconnecting [ 4492.785487] Lustre: 139309:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4496.759703] Lustre: *** cfs_fail_loc=159, val=0*** [ 4496.769394] Lustre: Skipped 3 previous similar messages [ 4512.385148] Lustre: lustre-MDT0000: Client 7305d9f8-58fe-4752-8019-9135bb306e61 (at 192.168.202.4@tcp) reconnecting [ 4514.337077] Lustre: *** cfs_fail_loc=159, val=0*** [ 4514.345238] Lustre: Skipped 14 previous similar messages [ 4530.747234] Lustre: lustre-MDT0000: Client 7305d9f8-58fe-4752-8019-9135bb306e61 (at 192.168.202.4@tcp) reconnecting [ 4532.994635] Lustre: 139606:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4540.897746] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4540.907465] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4540.929862] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4542.954065] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4542.958384] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4542.973735] LustreError: Skipped 1 previous similar message [ 4543.004263] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4545.915896] Lustre: server umount lustre-OST0000 complete [ 4552.164574] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4552.169667] LustreError: Skipped 1 previous similar message [ 4552.171961] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4552.184456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4552.187085] Lustre: Skipped 2 previous similar messages [ 4556.619129] Lustre: server umount lustre-MDT0000 complete [ 4558.820913] LustreError: 138951:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4558.849269] LustreError: 138951:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4560.959577] LustreError: 139556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786010642 with bad export cookie 6904073076957760463 [ 4560.962893] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4560.988248] LustreError: 139556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4561.565137] Lustre: server umount lustre-MDT0001 complete [ 4574.105938] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 4577.179750] Key type lgssc unregistered [ 4577.492434] LNet: 140669:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4577.498233] LNetError: 140669:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4577.528250] LNet: Removed LNI 192.168.202.104@tcp [ 4578.597779] Key type .llcrypt unregistered [ 4578.600760] Key type ._llcrypt unregistered [ 4605.907970] Key type ._llcrypt registered [ 4605.910271] Key type .llcrypt registered [ 4606.070444] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 4608.051114] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4608.078375] alg: No test for adler32 (adler32-zlib) [ 4609.112139] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 4609.392256] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 4611.073811] Key type lgssc registered [ 4612.754757] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4622.785742] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4622.853368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4624.454993] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4630.219705] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4634.599287] LustreError: 142086:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4634.627910] LustreError: 142086:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4639.713854] LustreError: 142085:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4641.827228] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4642.355215] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4647.624409] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4657.695507] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4664.113196] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4672.665392] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4673.273887] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4680.122729] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4687.606670] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4689.053465] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4694.782919] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4699.488863] Lustre: 144672:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4702.396971] Lustre: 144722:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4704.117489] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4705.434820] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4709.781303] Lustre: 144869:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4710.971106] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4712.287464] Lustre: 144967:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4718.566271] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4718.584949] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4718.612103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4720.620863] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4720.631458] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4720.631712] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4720.664484] Lustre: Skipped 1 previous similar message [ 4724.514808] Lustre: server umount lustre-OST0000 complete [ 4730.851763] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4730.864993] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4730.865399] Lustre: Skipped 1 previous similar message [ 4730.894429] Lustre: Skipped 1 previous similar message [ 4735.973025] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4736.379470] Lustre: server umount lustre-MDT0000 complete [ 4741.094491] LustreError: 142813:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4741.133575] LustreError: 142813:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4741.924305] LustreError: 142820:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786010823 with bad export cookie 3394540864114550836 [ 4741.938137] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4741.942718] LustreError: 142820:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4742.206857] Lustre: server umount lustre-MDT0001 complete [ 4760.179614] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 4764.812781] Key type lgssc unregistered [ 4765.190043] LNet: 146031:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4765.200861] LNetError: 146031:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4765.216176] LNet: Removed LNI 192.168.202.104@tcp [ 4766.526977] Key type .llcrypt unregistered [ 4766.533795] Key type ._llcrypt unregistered [ 4797.759219] Key type ._llcrypt registered [ 4797.761053] Key type .llcrypt registered [ 4797.909810] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 4799.280662] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4799.401704] alg: No test for adler32 (adler32-zlib) [ 4800.640614] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 4800.935962] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 4802.762059] Key type lgssc registered [ 4803.974736] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4812.463312] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4812.494175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4814.063459] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4819.879846] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4823.541344] LustreError: 147447:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4823.550877] LustreError: 147447:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4828.653906] LustreError: 147448:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4829.916514] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4830.521820] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4835.817690] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4845.223792] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4853.031696] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4860.877658] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4861.275517] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4869.540292] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 4875.633205] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4879.513566] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4883.864486] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4892.505314] Lustre: *** cfs_fail_loc=159, val=0*** [ 4908.163246] Lustre: lustre-MDT0000: Client a6f72115-2085-4924-acde-54ab19502268 (at 192.168.202.4@tcp) reconnecting [ 4916.707539] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4916.711222] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4916.745321] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4919.780852] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4919.781419] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4922.537937] Lustre: server umount lustre-OST0000 complete [ 4927.970224] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4927.983170] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4927.998976] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4928.007358] Lustre: Skipped 1 previous similar message [ 4930.017788] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4930.034976] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4933.595609] Lustre: server umount lustre-MDT0000 complete [ 4935.139817] LustreError: 148439:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4939.290251] LustreError: 147429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786011021 with bad export cookie 5939467763990209258 [ 4939.298409] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4939.302224] LustreError: 147429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4940.182869] Lustre: server umount lustre-MDT0001 complete [ 4954.336279] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 4958.236744] Key type lgssc unregistered [ 4958.647766] LNet: 151138:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4958.653694] LNetError: 151138:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4958.667607] LNet: Removed LNI 192.168.202.104@tcp [ 4959.649791] Key type .llcrypt unregistered [ 4959.653558] Key type ._llcrypt unregistered [ 4989.045635] Key type ._llcrypt registered [ 4989.048682] Key type .llcrypt registered [ 4989.286975] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 4991.296715] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4991.330113] alg: No test for adler32 (adler32-zlib) [ 4992.631915] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 4992.926641] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 4994.576805] Key type lgssc registered [ 4996.206649] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5006.019404] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5006.066900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5007.674598] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5012.978855] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5017.061049] LustreError: 152554:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5017.098044] LustreError: 152554:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5022.180913] LustreError: 152555:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5023.889876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5024.602466] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5029.836579] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5038.072879] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5044.884618] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5053.682978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5054.314344] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5062.939690] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5069.684164] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5071.445984] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5077.931548] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5098.059146] Lustre: lustre-OST0000: evicting 4c737612-5cfe-4ffe-ae17-09164f15a6de (at 192.168.202.4@tcp) by administrative request [ 5098.084674] Lustre: Skipped 1 previous similar message [ 5129.185649] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5129.197773] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5129.200704] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5129.224457] Lustre: Skipped 1 previous similar message [ 5134.317430] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5134.328679] Lustre: Skipped 2 previous similar messages [ 5139.428151] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5140.962763] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5141.119174] Lustre: server umount lustre-OST0000 complete [ 5147.619202] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5147.629849] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5147.650542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5147.653238] Lustre: Skipped 1 previous similar message [ 5149.679466] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5151.875347] Lustre: server umount lustre-MDT0000 complete [ 5154.785291] LustreError: 152555:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5154.817413] LustreError: 152555:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5155.751225] LustreError: 154409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786011237 with bad export cookie 7335797438777329742 [ 5155.764426] LustreError: 154409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5155.767885] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5156.273428] Lustre: server umount lustre-MDT0001 complete [ 5168.459251] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 5172.018357] Key type lgssc unregistered [ 5172.311021] LNet: 156633:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5172.319494] LNetError: 156633:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5172.333640] LNet: Removed LNI 192.168.202.104@tcp [ 5173.463640] Key type .llcrypt unregistered [ 5173.465669] Key type ._llcrypt unregistered [ 5265.352323] Key type ._llcrypt registered [ 5265.357347] Key type .llcrypt registered [ 5265.427867] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 5265.980095] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5265.990186] alg: No test for adler32 (adler32-zlib) [ 5267.170799] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 5267.614500] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 5269.370060] Key type lgssc registered [ 5270.907741] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5278.502910] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5278.524965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5279.904869] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5284.428458] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5289.448379] LustreError: 159131:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5289.463356] LustreError: 159131:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5293.118163] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5293.499519] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5298.124576] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5305.755762] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5311.109295] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5318.170327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5318.510044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5324.674942] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5325.139942] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5331.696361] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5338.997184] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5349.345725] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5349.360045] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5349.362377] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5349.376169] Lustre: Skipped 1 previous similar message [ 5349.387947] Lustre: Skipped 1 previous similar message [ 5354.472226] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5354.478662] Lustre: Skipped 1 previous similar message [ 5355.411911] Lustre: server umount lustre-OST0000 complete [ 5360.102445] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5360.113337] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5360.137132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5364.708682] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5364.711955] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5364.730518] Lustre: Skipped 1 previous similar message [ 5366.135375] Lustre: server umount lustre-MDT0000 complete [ 5369.849608] LustreError: 159861:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5369.889451] LustreError: 159861:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5370.441814] LustreError: 159110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786011452 with bad export cookie 3766861511061862966 [ 5370.457671] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5370.458344] LustreError: 159110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5371.039162] Lustre: server umount lustre-MDT0001 complete [ 5383.197260] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 5386.204889] Key type lgssc unregistered [ 5386.472756] LNet: 162698:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5386.489729] LNetError: 162698:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5387.558592] LNet: Removed LNI 192.168.202.104@tcp [ 5388.413533] Key type .llcrypt unregistered [ 5388.417522] Key type ._llcrypt unregistered [ 5414.341656] Key type ._llcrypt registered [ 5414.343676] Key type .llcrypt registered [ 5414.429488] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 5415.625940] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5415.698843] alg: No test for adler32 (adler32-zlib) [ 5416.832380] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 5417.094427] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 5418.784259] Key type lgssc registered [ 5419.930507] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5437.045636] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 06:18:36 (1786011516) [ 5455.039105] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 5466.809846] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 5508.643376] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5508.705627] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5510.019536] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5510.044491] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5510.138669] Lustre: lustre-MDT0000: new disk, initializing [ 5510.233113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5510.250327] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5514.970905] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5526.954191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5527.147408] Lustre: 166966:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5527.228738] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5527.233809] Lustre: Skipped 1 previous similar message [ 5527.336390] Lustre: lustre-MDT0001: new disk, initializing [ 5527.458408] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5527.544978] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5527.568566] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5532.749896] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5543.052304] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5549.020641] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5556.469106] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5556.729930] Lustre: lustre-OST0000: new disk, initializing [ 5556.733294] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5556.745705] Lustre: 168936:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5556.868687] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5564.410480] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5566.036602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5566.044141] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5566.199692] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5574.921655] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5586.916907] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5586.929036] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5586.934752] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5586.943970] Lustre: Skipped 1 previous similar message [ 5591.674796] Lustre: server umount lustre-OST0000 complete [ 5597.159450] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5597.168258] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5597.182696] Lustre: Skipped 2 previous similar messages [ 5602.202652] Lustre: server umount lustre-MDT0000 complete [ 5605.903190] LustreError: 166958:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786011687 with bad export cookie 13663717680340003521 [ 5605.906425] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5605.923808] LustreError: 166958:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5606.282555] Lustre: server umount lustre-MDT0001 complete [ 5618.235921] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 5621.054169] Key type lgssc unregistered [ 5621.294778] LNet: 170847:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5621.306557] LNetError: 170847:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5622.379414] LNet: Removed LNI 192.168.202.104@tcp [ 5623.128730] Key type .llcrypt unregistered [ 5623.130583] Key type ._llcrypt unregistered [ 5649.159847] Key type ._llcrypt registered [ 5649.163389] Key type .llcrypt registered [ 5649.263357] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 5650.777655] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5650.811326] alg: No test for adler32 (adler32-zlib) [ 5651.826804] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 5652.024677] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 5653.696599] Key type lgssc registered [ 5654.738799] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5661.908222] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5661.919466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5663.283824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5666.987801] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5673.442835] LustreError: 172264:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5673.470430] LustreError: 172264:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5676.085564] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5676.420878] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5681.250470] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5689.770964] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5695.752721] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5703.466194] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5703.793257] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5708.934887] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5710.844357] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 5719.211434] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5730.241156] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5730.317054] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5730.369544] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5730.390631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.409829] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5732.020689] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5732.080849] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5732.108751] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5732.124184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.139046] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5733.455324] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5733.478049] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5733.491521] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5733.501650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.511026] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5734.834678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.837731] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5735.844983] Lustre: Skipped 1 previous similar message [ 5735.868387] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5735.885421] Lustre: Skipped 1 previous similar message [ 5735.892104] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5735.896779] Lustre: Skipped 1 previous similar message [ 5735.903081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.919932] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5735.923059] Lustre: Skipped 1 previous similar message [ 5737.117431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.265384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.202327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.235513] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5740.253766] Lustre: Skipped 3 previous similar messages [ 5740.275706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5740.290056] Lustre: Skipped 3 previous similar messages [ 5740.298519] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5740.305513] Lustre: Skipped 3 previous similar messages [ 5740.310089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.325924] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5740.337033] Lustre: Skipped 3 previous similar messages [ 5741.414337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.191793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.259326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.523162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.521522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.649113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.974937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.200925] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5749.220790] Lustre: Skipped 7 previous similar messages [ 5749.251540] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5749.269543] Lustre: Skipped 7 previous similar messages [ 5749.285673] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5749.291044] Lustre: Skipped 7 previous similar messages [ 5749.296948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.317052] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5749.331674] Lustre: Skipped 7 previous similar messages [ 5750.355961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.347907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.495655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.575407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.344878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.354495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.491293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.803688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.880670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.675813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.611527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.544688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.428993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.342079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.305382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.059283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.814180] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5765.820550] Lustre: Skipped 16 previous similar messages [ 5765.835821] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5765.845740] Lustre: Skipped 16 previous similar messages [ 5765.850165] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5765.852587] Lustre: Skipped 16 previous similar messages [ 5765.855603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.864071] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5765.866962] Lustre: Skipped 16 previous similar messages [ 5766.674562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.607827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.740312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.723856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.623529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.552239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.609799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.626221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.523779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.406316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.552312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.701300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.687100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.732703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.571103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.484180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.420508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.323542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.217668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.120301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.093849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.032167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.960623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.843649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.954656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.104532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.225665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.289930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.408213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.454101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.461890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.814747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.924601] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5798.933297] Lustre: Skipped 32 previous similar messages [ 5798.954731] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5798.961724] Lustre: Skipped 32 previous similar messages [ 5798.964856] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5798.967777] Lustre: Skipped 32 previous similar messages [ 5798.972699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.981860] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5798.989743] Lustre: Skipped 32 previous similar messages [ 5800.003213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.177869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.281525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.414389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.564839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.311689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.200065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.121985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.061942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.909894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.912068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.769130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.629777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.546912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.411939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.205401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.053977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.089450] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5816.100220] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5816.106667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.156460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.973405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.899203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.949753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.024938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.880915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.059673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.078383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.172470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.337477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.212627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.197429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.251599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.157822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.089166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.310463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.216799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.267977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.139740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.260928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.164416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.143612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.997613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.982840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.782975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.816967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.791265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.728671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.699464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.596447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.541588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.424957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.215460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.115025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.089829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.961620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.872683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.756530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.668705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.407964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.129766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.080058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.966915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.945166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.787726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.715954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.699958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.493056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.493969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.345725] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5863.348490] Lustre: Skipped 67 previous similar messages [ 5863.354481] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5863.361937] Lustre: Skipped 67 previous similar messages [ 5863.368912] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5863.371790] Lustre: Skipped 67 previous similar messages [ 5863.377511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.399264] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5863.404594] Lustre: Skipped 67 previous similar messages [ 5864.366582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.240990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.421130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.672349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.810438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.015948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.119039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.396326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.647824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.689801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.899039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.323605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.753387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.842218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.880607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.856637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.058209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.913592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.500972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.498673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.711132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.152748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.616133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.549908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.859109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.118696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.414116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.449251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.650971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.150070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.643154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.058409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.280479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.567743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.983767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.253935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.614120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.742468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.904637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.107737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.050888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.159881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.177682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.469443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.051804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.928454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.819361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.676757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.614329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.326327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.168644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.940567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.806943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.884579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.047302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.063988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.968818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.705463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.962584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.885772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.879066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.681987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.950893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.057128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.298035] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5937.303740] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5937.321743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.605762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.005752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.999441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.210543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.715177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.044517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.296289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.584667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.052458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.277550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.782848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.095175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.556858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.171884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.840159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.332963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.523774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.874320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.477519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.620550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.885235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.390355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.665604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.011238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.348816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.341458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.774906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.186028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.636331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.890186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.126644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.202086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.397681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.576577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.779959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.921585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.948987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.235856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.558903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.665197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.563719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.426467] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5991.435982] Lustre: Skipped 106 previous similar messages [ 5991.449958] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5991.457237] Lustre: Skipped 106 previous similar messages [ 5991.461619] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5991.466733] Lustre: Skipped 106 previous similar messages [ 5991.470483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.493319] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5991.497168] Lustre: Skipped 106 previous similar messages [ 5992.467556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.648810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.703854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.526913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.572952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.546032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.548494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.449266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.628777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.043236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.122270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.165537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.228188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.164584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.178442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.337412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.321402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.279543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.338506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.332564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.215559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.297179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.219270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.217401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.166337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.208378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.653123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.767391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.892577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.943913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.961759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.138167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.011666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.086064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.976311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.102036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.042279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.146197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.100695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.960712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.803982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.770569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.671326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.643336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.356130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.340735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.115324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.119204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.115655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.096110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.875247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.810658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.558322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.492392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.460346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.613074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.728063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.868395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.787124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.828203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.915616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.888735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.127622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.101368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.337405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.513459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.545532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.908526] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6060.923772] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6060.936488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.137836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.190884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.982823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.030638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.041170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.083548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.006810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.286140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.451262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.643961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.935330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.015084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.034043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.908874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.811523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.884811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.047944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.861707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.075918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.385425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.471645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.546243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.785656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.244271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.673607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.068468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.378189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.713132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.070688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.844266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.366843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.460560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.505358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.915359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.943091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.438316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.499540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.557247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.768465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.860870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.144315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.363371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.679115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.848146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.939656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.013500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.835269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.864914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.912738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.931311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.102795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.309893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.382288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.294638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.489692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.506211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.422680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.404177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.319962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.474601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.422721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.662764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.642094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.744665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.701530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.781994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.951270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.066935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.115886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.504905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.807245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.021719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.266637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.556504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.488901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.613576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.622521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.769987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.752165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.865164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.905137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.934467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.057110] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6155.074950] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6155.091180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.093877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.413454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.246939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.300436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.480792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.524132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.725721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.510122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.390211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.214097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.428488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.504324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.795232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.002788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.025504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.330583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.488856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.814522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.923514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.225974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.537937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.196592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.340381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.286246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.508841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.680898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.980608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.209774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.109370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.457714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.794189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.223086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.383879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.314148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.297417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.429510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.890808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.303094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.798394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.188320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.555705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.840912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.053648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.364122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.487565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.719543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.847300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.278199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.705613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.696562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.759561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.022055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.291136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.199275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.426523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.878541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.973781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.072546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.120635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.765099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.639112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.532667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.491323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.730324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.645087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.569707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.576904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.824413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.799872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.869588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.904592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.928496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.640812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.487032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.847852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.164619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.127688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.466270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.511760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.705233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.821743] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6247.835702] Lustre: Skipped 231 previous similar messages [ 6247.868272] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6247.887488] Lustre: Skipped 231 previous similar messages [ 6247.894209] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6247.901993] Lustre: Skipped 231 previous similar messages [ 6247.910464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.927555] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6247.938776] Lustre: Skipped 231 previous similar messages [ 6249.334819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.565221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.470984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.380679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.490982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.482143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.736737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.633118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.762398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.836294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.866597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.923744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.914933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.975807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.011886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.035297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.032295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.045315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.885722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.871057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.836467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.938741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.933709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.896126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.896422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.978492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.035186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.968741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.939712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.065187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.070474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.071077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.930637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.071836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.997460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.801966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.808483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.688339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.701545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.872623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.845059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.883232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.765953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.728616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.548848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.443583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.414721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.394526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.315123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.297674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.220127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.078022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.056994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.146288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.143715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.003219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.874850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.892309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.776921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.596667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.451798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.447982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.348620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.335782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.270741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.236803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.205527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.239771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.229286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.981437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.107406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.074230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.262895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.073114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.082484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.021585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.989736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.890551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.781470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.666301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.410555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.116516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.263163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.155955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.803465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.551989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.744844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.710995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.925669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.198309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.599694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.960089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.246228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.626528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.669712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.787313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.735342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.546516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.676978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.485498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.518300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.927076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.048647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.310437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.682721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.957564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.463939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.598736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.888031] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6358.903942] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6358.928513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.152481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.692525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.658636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.473065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.845876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.065470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.258299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.461090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.764115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.675301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.814224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.757850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.865391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.661619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.653902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.556884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.491839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.768494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.862231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.780567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.752234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.809288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.759416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.885690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.880100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.913237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.779953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.782789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.549031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.553656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.599177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.504408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.439173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.515387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.403529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.443805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.706279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.673953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.746693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.720788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.691066] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6400.702046] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6400.716275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.848840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.966149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.181446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.028504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.937085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.054644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.182618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.111353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.202425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.982422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.953320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.947301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.978850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.859435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.932159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.955640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.410586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.432178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.505833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.056598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.136606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.222441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.197720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.453962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.635929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.933960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.150753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.214502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.112635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.479303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.562705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.178991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.612818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.809817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.878197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.034651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.031230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.869624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.882158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.051621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.870290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.305907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.583187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.851334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.338996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.027535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.376489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.993288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.608741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.822614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.811557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.913581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.539753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.181678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.257299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.562745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.038507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.113084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.039883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.036398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.582189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.709539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.108681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.668092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.688436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.726610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.655891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.637651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.475931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.523797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.430303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.678840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.497914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.355613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.118793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.226612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.187688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.140221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.076360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.986082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.122042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.137480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.012508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.019727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.994270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.037781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.150680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.106378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.139852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.035861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.373369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.514149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.576094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.411279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.328841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.579758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.620327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.963488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.196242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.505989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.722137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.598273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.707775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.865435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.953283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.879493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.088982] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6520.098972] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6520.116419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.954583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.101062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.040778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.357856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.622926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.004589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.462500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.142532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.377505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.463681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.768837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.178427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.709738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.877790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.409541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.193174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.022442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.861075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.947910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.026755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.951921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.948738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.840745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.693753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.363807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.344923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.256962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.066539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.828730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.704614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.719090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.858628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.945855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.070126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.135050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.464879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.322503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.332891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.426196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.363559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.538581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.534517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.837792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.140487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.350582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.235603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.555800] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6571.567984] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6571.574698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.889612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.513864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.726200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.971086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.288463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.184967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.052551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.196957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.989480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.134928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.435481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.807609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.211907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.292698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.627660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.810909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.487117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.736195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.186731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.440137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.099652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.113221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.568306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.720573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.579756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.891684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.948495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.861879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.931842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.657438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.610561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.647806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.600235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.796082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.673644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.742741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.635170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.629588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.530958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.541791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.898314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.773351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.734399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.376361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.146093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.191915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.136872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.387593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.323384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.103855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.115122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.225881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.945485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.636690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.866706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.054235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.394328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.290732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.569287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.739569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.160492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.124215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.270645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.355341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.499523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.666799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.796319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.937301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.648976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.579860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.498494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.405905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.325122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.378516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.169807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.289883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.460581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.394970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.629269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.648993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.637274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.735113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.888961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.138732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.314547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.396322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.448513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.455816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.410700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.368980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.133952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.043684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.885700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.673647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.582746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.491489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.364324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.380937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.168920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.215137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.215254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.277091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.233587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.212515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.045233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.073288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.100651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.158615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.079227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.930494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.854071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.806467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.557981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.451704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.265154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.162402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.019305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.114653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.028331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.938854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.862544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.849480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.817538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.725064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.492529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.198918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.004984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.893246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.800414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.704822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.661068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.623801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.528718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.397726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.372648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.265644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.077713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.919680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.990702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.963078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.962231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.912302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.100324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.003302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.897591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.998080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.924696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.902022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.050023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.936936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.872189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.906451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.084106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.026511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.312879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.168192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.136294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.082461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.059928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.200396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.355815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.458480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.597766] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6739.605349] LustreError: 172283:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6739.626779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.643959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.735655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.733862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.937026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.923813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.198743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.444315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.457057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.569847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.534842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.738393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.848905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.867848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.005921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.951929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.958212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.198937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.181758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.247348] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6760.251217] Lustre: Skipped 485 previous similar messages [ 6760.270746] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6760.280182] Lustre: Skipped 485 previous similar messages [ 6760.284298] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6760.291205] Lustre: Skipped 485 previous similar messages [ 6760.294850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.304483] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6760.312371] Lustre: Skipped 485 previous similar messages [ 6761.463562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.559374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.650991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.856485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.985075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.444087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.310407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.606400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.587347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.695774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.786610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.762710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.727545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.593901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.449619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.371764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.475734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.434535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.653189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.803783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.066522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.312443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.377627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.510986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.395349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.393729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.372898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.478445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.265410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.019792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.039995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.037565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.103267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.068437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.987433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.837935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6798.656465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.485895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.504196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.336517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7215.336232] LustreError: 174085:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7215.346747] LustreError: 172283:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7971.300061] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7971.308606] Lustre: Skipped 40 previous similar messages [ 7971.313609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7971.317149] Lustre: Skipped 1 previous similar message [ 7973.408950] Lustre: server umount lustre-OST0000 complete [ 7980.516374] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7980.531415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7981.540610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7981.550098] Lustre: Skipped 1 previous similar message [ 7986.658451] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7986.662028] Lustre: Skipped 1 previous similar message [ 7991.778397] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7991.798510] Lustre: Skipped 1 previous similar message [ 7992.288553] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7992.799673] Lustre: server umount lustre-MDT0000 complete [ 7996.907726] LustreError: 226122:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7996.938624] LustreError: 226122:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7996.954432] LustreError: 172245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786014078 with bad export cookie 17204060967142961794 [ 7996.970972] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7997.419097] Lustre: server umount lustre-MDT0001 complete [ 8013.169967] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 8016.940632] Key type lgssc unregistered [ 8017.316723] LNet: 227570:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8017.320981] LNetError: 227570:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8017.338482] LNet: Removed LNI 192.168.202.104@tcp [ 8018.564961] Key type .llcrypt unregistered [ 8018.567212] Key type ._llcrypt unregistered [ 8053.983516] Key type ._llcrypt registered [ 8053.985731] Key type .llcrypt registered [ 8054.177223] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8055.652900] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8055.712982] alg: No test for adler32 (adler32-zlib) [ 8056.853593] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 8057.105336] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 8058.864395] Key type lgssc registered [ 8060.327960] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8069.695892] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8069.734157] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8071.319880] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8076.580905] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8081.902712] LustreError: 228990:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8081.947019] LustreError: 228990:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8086.293483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8087.023930] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8094.178268] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8104.479436] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8110.748251] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8119.887434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8120.316587] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8128.342280] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8137.038317] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8141.831682] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8147.478412] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8150.458162] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8150.495652] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8150.509688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8150.537157] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8150.537845] LustreError: 229010:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8157.175583] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8157.181369] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8157.215836] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8157.239013] Lustre: Skipped 1 previous similar message [ 8158.273505] Lustre: lustre-OST0000: Not available for connect from 192.168.202.4@tcp (stopping) [ 8160.137374] Lustre: server umount lustre-OST0000 complete [ 8170.592146] LustreError: 229010:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8195.654045] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8195.945561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8198.003512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8198.041484] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8198.043672] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8198.075816] Lustre: Skipped 1 previous similar message [ 8204.940760] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8216.288642] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8217.890435] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8226.902559] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff940b10bf5000.ost_server_uuid 50 [ 8229.077361] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff940b10bf5000.ost_server_uuid in IDLE state after 0 sec [ 8239.101684] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8239.132279] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8239.165174] Lustre: Skipped 1 previous similar message [ 8239.176232] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8239.184821] Lustre: Skipped 1 previous similar message [ 8241.649806] Lustre: server umount lustre-OST0000 complete [ 8249.339231] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8249.347375] Lustre: Skipped 1 previous similar message [ 8249.351987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8249.359449] Lustre: Skipped 1 previous similar message [ 8253.294667] Lustre: server umount lustre-MDT0000 complete [ 8254.438026] LustreError: 231492:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8254.457177] LustreError: 231492:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8258.021040] LustreError: 228971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786014339 with bad export cookie 11947422648744981217 [ 8258.024278] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8258.067615] LustreError: 228971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8258.515800] Lustre: server umount lustre-MDT0001 complete [ 8274.266566] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 8277.778948] Key type lgssc unregistered [ 8278.277514] LNet: 234209:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8278.284563] LNetError: 234209:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8278.317742] LNet: Removed LNI 192.168.202.104@tcp [ 8279.750365] Key type .llcrypt unregistered [ 8279.752959] Key type ._llcrypt unregistered [ 8319.432836] Key type ._llcrypt registered [ 8319.434989] Key type .llcrypt registered [ 8319.583122] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8321.508787] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8321.539269] alg: No test for adler32 (adler32-zlib) [ 8322.808896] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 8323.100446] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 8324.888799] Key type lgssc registered [ 8326.131212] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8340.505405] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8340.545988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8342.220865] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8347.957886] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8352.739261] LustreError: 235941:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8352.745496] LustreError: 235941:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8357.879545] LustreError: 235940:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8359.271951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8359.718248] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8367.052925] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8370.652502] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8378.441548] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8378.889209] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8386.330373] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8387.124804] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8396.441070] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8396.743361] Lustre: lustre-OST0001: new disk, initializing [ 8396.752751] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8396.778727] Lustre: 238258:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8396.987426] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8403.799910] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8406.243355] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8406.257123] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8406.354547] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8416.004659] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8422.725250] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8441.830427] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8441.847414] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8441.870744] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8442.356499] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8442.382723] Lustre: Skipped 2 previous similar messages [ 8442.385132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8442.403528] Lustre: Skipped 3 previous similar messages [ 8445.674183] Lustre: server umount lustre-MDT0000 complete [ 8447.462572] LustreError: 235935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8447.483857] LustreError: 235935:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8450.727183] LustreError: 239024:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786014532 with bad export cookie 6452322550950874969 [ 8450.742829] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8450.761174] LustreError: 239024:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8451.071247] Lustre: server umount lustre-MDT0001 complete [ 8465.859890] Lustre: server umount lustre-OST0000 complete [ 8468.832618] Lustre: 235203:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786014534/real 1786014534] req@ffff962b027f1f80 x1872771840837248/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786014550 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8468.916736] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8471.653576] Lustre: server umount lustre-OST0001 complete [ 8498.647945] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8511.332157] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8512.064950] LustreError: 241364:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8512.223101] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8517.613022] LustreError: 241365:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8518.893755] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8527.846276] LustreError: 241364:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8527.887237] LustreError: 241364:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8531.044925] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8531.660663] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8538.684192] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8542.003608] Lustre: 242507:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8550.314095] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8550.886961] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8556.074268] LustreError: 242861:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8558.138058] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 8559.390195] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8574.511750] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8580.145255] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 8582.632630] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8591.127876] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8595.507240] Lustre: 244378:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8610.812773] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8610.816883] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8610.859666] Lustre: Skipped 3 previous similar messages [ 8610.885118] Lustre: Skipped 3 previous similar messages [ 8615.442914] Lustre: server umount mylustre-MDT0000 complete [ 8615.935262] LustreError: 241360:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8615.994351] LustreError: 241360:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 8620.407792] LustreError: 241341:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786014702 with bad export cookie 6452322550950877741 [ 8620.419209] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8620.433565] LustreError: 241341:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8621.027222] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8621.079266] Lustre: server umount mylustre-MDT0001 complete [ 8626.746734] Lustre: server umount mylustre-OST0000 complete [ 8632.310571] Lustre: server umount mylustre-OST0001 complete [ 8661.017311] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8673.910308] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8674.554537] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8674.564933] Lustre: Skipped 1 previous similar message [ 8679.552425] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8685.027691] LustreError: 246689:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8685.086524] LustreError: 246689:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 8692.258393] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8702.431399] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8706.056952] Lustre: 247831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8712.050200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8719.815391] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 8720.365919] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8731.127715] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8736.814749] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 8739.873921] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8750.529199] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8754.378615] Lustre: 249703:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8762.338766] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8762.353692] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8762.377891] Lustre: Skipped 3 previous similar messages [ 8762.415209] Lustre: Skipped 3 previous similar messages [ 8764.083375] Lustre: server umount tfs-MDT0000 complete [ 8767.957666] LustreError: 247833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786014849 with bad export cookie 6452322550950879631 [ 8767.961602] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8767.982307] LustreError: 247833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8769.138170] Lustre: server umount tfs-MDT0001 complete [ 8784.354355] Lustre: server umount tfs-OST0000 complete [ 8799.203185] Lustre: server umount tfs-OST0001 complete [ 8826.269608] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 8837.923644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8838.466797] LustreError: 251954:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8838.496675] LustreError: 251954:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 8838.575784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8838.580279] Lustre: Skipped 3 previous similar messages [ 8844.286863] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8854.927890] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8862.625536] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8866.540790] Lustre: 253096:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8874.946537] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8884.059725] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 8884.604566] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8896.380570] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8901.645953] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 8903.517447] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 8913.292741] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8917.454574] Lustre: 254970:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8925.374084] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 07:16:44 (1786015004) [ 8942.567725] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8942.571941] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8942.577852] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8942.615596] Lustre: Skipped 4 previous similar messages [ 8949.252578] Lustre: server umount lustre-MDT0000 complete [ 8953.814620] LustreError: 251930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786015035 with bad export cookie 6452322550950881297 [ 8953.824015] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8957.953311] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8957.971641] Lustre: Skipped 8 previous similar messages [ 8968.160306] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8968.442336] Lustre: server umount lustre-MDT0001 complete [ 8987.107669] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8987.317995] Lustre: server umount lustre-OST0000 complete [ 8998.805070] Lustre: server umount lustre-OST0001 complete [ 9007.287415] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 9021.148470] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9068.176841] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 9068.176841] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 9068.176841] [ 9068.225387] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 9068.566412] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9068.645839] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9068.775479] Lustre: lustre-MDT0000: new disk, initializing [ 9068.992775] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9069.006415] Lustre: Skipped 3 previous similar messages [ 9069.026595] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9074.283363] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9086.415282] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 9086.415282] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 9086.415282] [ 9086.473707] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 9086.603550] Lustre: 258160:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9086.710138] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9086.714550] Lustre: Skipped 1 previous similar message [ 9086.789782] Lustre: lustre-MDT0001: new disk, initializing [ 9086.926367] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9086.944987] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9092.979398] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9105.543174] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9105.826876] Lustre: lustre-OST0000: new disk, initializing [ 9105.830273] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9105.842342] Lustre: 259743:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9107.601479] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9107.620276] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9107.770232] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9113.759818] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9127.296561] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9127.485942] Lustre: lustre-OST0001: new disk, initializing [ 9127.491327] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9127.505288] Lustre: 260615:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9129.490661] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9129.512638] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9129.688876] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9135.861557] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9149.291466] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 07:20:28 (1786015228) [ 9167.790432] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 07:20:47 (1786015247) [ 9184.739826] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9184.743837] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9184.755074] LustreError: Skipped 1 previous similar message [ 9184.761229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9184.781844] Lustre: Skipped 3 previous similar messages [ 9184.804558] Lustre: Skipped 5 previous similar messages [ 9190.131276] Lustre: server umount lustre-OST0000 complete [ 9191.404362] LustreError: 259725:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9191.453720] LustreError: 259725:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 21 previous similar messages [ 9209.314161] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9209.727983] Lustre: server umount lustre-MDT0000 complete [ 9214.411467] LustreError: 258152:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786015296 with bad export cookie 6452322550950882851 [ 9214.424616] LustreError: 258152:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 9214.430585] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9217.009177] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9217.013650] Lustre: Skipped 11 previous similar messages [ 9221.051363] Lustre: server umount lustre-MDT0001 complete [ 9235.801750] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 9309.839693] Lustre: server umount lustre-OST0001 complete [ 9318.642238] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [ 9336.155855] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9378.729965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9378.939501] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9378.963910] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9379.021583] Lustre: lustre-MDT0000: new disk, initializing [ 9379.077993] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9379.082483] Lustre: Skipped 3 previous similar messages [ 9379.092447] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9384.620056] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9396.464246] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9396.620217] Lustre: 265411:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9396.696291] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9396.703822] Lustre: Skipped 1 previous similar message [ 9396.862880] Lustre: lustre-MDT0001: new disk, initializing [ 9397.051204] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9397.077776] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9401.981755] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9412.697661] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9420.127752] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9428.063908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9428.348848] Lustre: lustre-OST0000: new disk, initializing [ 9428.354860] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9428.367269] Lustre: 267384:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9429.715961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9429.735634] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9430.011906] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9436.266232] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9447.469974] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9455.602024] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9455.605630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9455.606509] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9455.615889] Lustre: Skipped 5 previous similar messages [ 9455.621779] Lustre: Skipped 1 previous similar message [ 9461.682270] Lustre: server umount lustre-OST0000 complete [ 9468.909103] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9468.925495] LustreError: Skipped 1 previous similar message [ 9480.160175] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9480.440438] Lustre: server umount lustre-MDT0000 complete [ 9484.677517] LustreError: 266217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786015566 with bad export cookie 6452322550950884321 [ 9484.684698] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9484.687669] LustreError: 266217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9485.001917] Lustre: server umount lustre-MDT0001 complete [ 9501.899676] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 9505.586281] Key type lgssc unregistered [ 9505.878594] LNet: 269148:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9505.891854] LNetError: 269148:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9505.916163] LNet: Removed LNI 192.168.202.104@tcp [ 9507.035231] Key type .llcrypt unregistered [ 9507.038114] Key type ._llcrypt unregistered [ 9537.984393] Key type ._llcrypt registered [ 9537.986878] Key type .llcrypt registered [ 9538.129718] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9539.594624] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9539.658551] alg: No test for adler32 (adler32-zlib) [ 9540.842156] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 9541.220602] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 9543.064194] Key type lgssc registered [ 9544.658559] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9553.836585] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9553.864856] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9555.543375] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9560.557503] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9565.666484] LustreError: 270612:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9565.698356] LustreError: 270612:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9569.821158] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9570.161650] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9575.794596] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9584.531297] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9591.629725] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9599.110742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9599.732132] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9600.782205] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9606.876593] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9614.981406] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9616.316352] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9620.449262] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9620.457722] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9620.469959] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9620.483857] Lustre: Skipped 1 previous similar message [ 9620.512479] Lustre: Skipped 1 previous similar message [ 9624.325973] Lustre: server umount lustre-OST0000 complete [ 9630.705647] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9630.721499] Lustre: Skipped 1 previous similar message [ 9630.722563] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9630.735240] Lustre: Skipped 1 previous similar message [ 9635.592088] Lustre: server umount lustre-MDT0000 complete [ 9635.822326] LustreError: 270613:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9635.857055] LustreError: 270613:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9640.585455] LustreError: 270593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786015722 with bad export cookie 7020411456639129507 [ 9640.596417] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9640.599553] LustreError: 270593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9641.007288] Lustre: server umount lustre-MDT0001 complete [ 9658.200354] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 9661.698052] Key type lgssc unregistered [ 9662.084960] LNet: 274219:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9662.090495] LNetError: 274219:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9662.122951] LNet: Removed LNI 192.168.202.104@tcp [ 9663.094111] Key type .llcrypt unregistered [ 9663.097125] Key type ._llcrypt unregistered [ 9684.534365] Key type ._llcrypt registered [ 9684.537607] Key type .llcrypt registered [ 9684.697634] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9686.179237] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9686.275128] alg: No test for adler32 (adler32-zlib) [ 9687.853246] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 9688.238660] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 9689.952532] Key type lgssc registered [ 9691.256997] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9702.688804] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9702.715911] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9718.497926] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9723.616430] LustreError: 275414:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.104@tcp: failed processing log, type 4: rc = -110 [ 9750.304218] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9758.346603] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [ 9767.901470] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9770.702388] Lustre: server umount lustre-OST0000 complete [ 9793.518250] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [ 9797.563462] Key type lgssc unregistered [ 9798.037232] LNet: 277050:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9798.046698] LNetError: 277050:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9799.090438] LNet: Removed LNI 192.168.202.104@tcp [ 9800.352860] Key type .llcrypt unregistered [ 9800.364213] Key type ._llcrypt unregistered [ 9844.389242] Key type ._llcrypt registered [ 9844.391316] Key type .llcrypt registered [ 9844.512242] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [ 9846.126698] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9846.187206] alg: No test for adler32 (adler32-zlib) [ 9847.376549] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [ 9847.646085] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [ 9849.352218] Key type lgssc registered [ 9850.861949] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9867.022026] loop0: detected capacity change from 0 to 204800000 [ 9867.276908] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9867.291440] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9867.894947] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9869.037469] loop0: detected capacity change from 0 to 204800000 [ 9869.264753] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9869.284978] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9869.848000] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9871.192681] loop0: detected capacity change from 0 to 204800000 [ 9871.506588] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9871.524513] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9871.624259] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9872.817420] loop0: detected capacity change from 0 to 204800000 [ 9873.049323] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9873.057707] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9873.150404] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9874.386813] loop0: detected capacity change from 0 to 204800000 [ 9874.426529] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9879.281703] loop0: detected capacity change from 0 to 204800000 [ 9879.312830] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9883.851714] loop0: detected capacity change from 0 to 204800000 [ 9883.871876] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9889.126261] loop0: detected capacity change from 0 to 204800000 [ 9889.154795] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9894.352697] loop0: detected capacity change from 0 to 204800000 [ 9894.500969] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9894.531891] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9894.661364] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9894.669638] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9894.682629] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9894.692197] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9894.713083] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9894.735781] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9894.742863] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9894.753898] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9894.765211] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9894.772513] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9894.779838] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9894.791052] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9894.807181] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9894.825559] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9900.948778] Lustre: server umount MGS complete [ 9902.316616] loop0: detected capacity change from 0 to 204800000 [ 9902.437021] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9902.866335] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9902.929833] Lustre: lustre-MDT0000: Aborting client recovery [ 9904.136669] loop1: detected capacity change from 0 to 204800000 [ 9904.346711] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9904.483307] Lustre: 280908:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9904.498596] Lustre: 280908:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9904.507309] Lustre: 280908:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9904.520622] Lustre: 280908:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9904.529632] Lustre: 280908:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9904.538709] Lustre: 280908:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9904.550145] Lustre: 280908:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9904.582674] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9904.758740] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9904.825292] Lustre: lustre-MDT0001: Aborting client recovery [ 9905.921931] loop2: detected capacity change from 0 to 204800000 [ 9906.202791] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9906.326580] Lustre: 280986:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9906.342870] Lustre: 280986:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9906.362981] Lustre: 280986:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9906.380430] Lustre: 280986:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9906.394486] Lustre: 280986:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9906.406705] Lustre: 280986:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9906.425168] Lustre: 280986:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9906.620058] Lustre: Found index 0 for lustre-OST0000, updating log [ 9906.654898] Lustre: 281005:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9906.772468] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9906.808637] Lustre: lustre-OST0000: Aborting recovery [ 9907.903647] loop3: detected capacity change from 0 to 204800000 [ 9908.066215] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9908.185485] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9908.200154] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9908.219970] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9908.228803] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9908.244394] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9908.270753] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9908.298895] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9908.315905] Lustre: Found index 1 for lustre-OST0001, updating log [ 9908.357350] Lustre: 281075:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9908.475926] Lustre: lustre-OST0001: Aborting recovery [ 9912.550621] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9912.566952] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9912.573105] LustreError: 280852:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9912.589431] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9912.626590] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9914.615805] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9914.632970] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9917.931894] LustreError: 280819:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 777us: rc = -2 [ 9918.445493] LustreError: 280820:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 9705us: rc = -2 [ 9918.482191] LustreError: 280820:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 14 previous similar messages [ 9935.328904] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9935.343961] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9935.355064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9935.375443] Lustre: Skipped 3 previous similar messages [ 9935.398252] Lustre: Skipped 3 previous similar messages [ 9938.180323] Lustre: server umount lustre-MDT0000 complete [ 9939.099416] LustreError: 280802:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786016021 with bad export cookie 435685267140203587 [ 9939.110416] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9940.450302] LustreError: 280816:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9940.451968] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9940.487287] LustreError: 280816:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9940.527757] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9940.544857] Lustre: Skipped 1 previous similar message [ 9944.481131] LustreError: 280817:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9944.481159] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9944.500475] Lustre: Skipped 1 previous similar message [ 9945.866289] Lustre: server umount lustre-MDT0001 complete [ 9953.356913] Lustre: server umount lustre-OST0000 complete [ 9960.544139] Lustre: server umount lustre-OST0001 complete [ 9970.374218] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 07:34:09 (1786016049) [10003.983735] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [10016.730322] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [10064.431170] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10064.784807] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10064.812984] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10064.908482] Lustre: lustre-MDT0000: new disk, initializing [10065.030543] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10065.043992] Lustre: Skipped 1 previous similar message [10065.057841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10071.980981] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10085.063083] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10085.300324] Lustre: 285146:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10085.400664] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10085.409104] Lustre: Skipped 1 previous similar message [10085.575861] Lustre: lustre-MDT0001: new disk, initializing [10085.743300] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10085.824199] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10085.843395] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10092.603263] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10105.833626] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10113.363308] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10122.558469] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10122.938843] Lustre: lustre-OST0000: new disk, initializing [10122.944259] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10122.962970] Lustre: 287114:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10123.107780] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10129.510395] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10129.531761] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10129.671204] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10130.053590] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10143.419434] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10150.020143] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10163.190094] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10173.754773] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10183.930557] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10227.173690] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10227.177750] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10227.200791] Lustre: Skipped 2 previous similar messages [10227.203447] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10229.642809] Lustre: server umount lustre-OST0000 complete [10234.339490] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10234.353741] LustreError: Skipped 1 previous similar message [10234.364188] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10234.396219] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10234.403402] Lustre: Skipped 1 previous similar message [10240.396334] Lustre: server umount lustre-MDT0000 complete [10242.533864] LustreError: 285155:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10242.545951] LustreError: 285155:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10245.647175] LustreError: 285137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786016327 with bad export cookie 435685267140206296 [10245.650552] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10245.657537] LustreError: 285137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10246.070445] Lustre: server umount lustre-MDT0001 complete [10256.753981] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10261.917447] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10267.282997] Lustre: server umount MGS complete [10280.280791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10280.717235] LustreError: 291182:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10280.774205] LustreError: 291195:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10280.848356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10286.062639] LustreError: 291183:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10286.253447] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10295.273454] LustreError: 291182:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10295.291405] LustreError: 291182:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10295.348970] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10302.087840] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10312.577669] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10320.404463] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10328.890144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10329.313784] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10329.330282] Lustre: Skipped 1 previous similar message [10336.345570] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10345.551640] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10386.917409] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10386.920040] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10386.932386] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10386.940952] Lustre: Skipped 2 previous similar messages [10386.963977] Lustre: Skipped 3 previous similar messages [10392.476474] Lustre: server umount lustre-OST0000 complete [10397.155087] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10403.326490] Lustre: server umount lustre-MDT0000 complete [10407.397336] LustreError: 291182:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10407.422573] LustreError: 291182:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10408.055924] LustreError: 291163:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786016490 with bad export cookie 435685267140208235 [10408.070444] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10408.471951] Lustre: server umount lustre-MDT0001 complete [10422.218616] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [10425.737562] Key type lgssc unregistered [10426.208403] LNet: 295136:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10426.228963] LNetError: 295136:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10426.277505] LNet: Removed LNI 192.168.202.104@tcp [10427.219348] Key type .llcrypt unregistered [10427.222664] Key type ._llcrypt unregistered [10474.105286] Key type ._llcrypt registered [10474.108212] Key type .llcrypt registered [10474.189562] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [10492.153324] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [10493.263790] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10493.351114] alg: No test for adler32 (adler32-zlib) [10494.619784] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [10494.915688] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [10496.640466] Key type lgssc registered [10498.352882] Lustre: Echo OBD driver; http://www.lustre.org/ [10541.199205] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10541.256820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10542.639591] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10542.670449] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10542.813411] Lustre: lustre-MDT0000: new disk, initializing [10542.929117] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10542.967927] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10548.459924] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10560.468460] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10560.569152] Lustre: 299095:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10560.614068] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10560.618051] Lustre: Skipped 1 previous similar message [10560.666405] Lustre: lustre-MDT0001: new disk, initializing [10560.770473] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10560.862414] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10560.870543] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10565.456587] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10577.120221] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10587.174471] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10595.521817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10595.802818] Lustre: lustre-OST0000: new disk, initializing [10595.807738] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10595.818217] Lustre: 301070:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10595.950864] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10604.425545] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10605.236870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10605.260803] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10605.442331] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10618.118969] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10625.527545] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10637.044678] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10642.064930] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10687.457556] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10687.460221] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10687.468903] LustreError: Skipped 1 previous similar message [10687.471624] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10687.507972] Lustre: Skipped 1 previous similar message [10691.798163] Lustre: server umount lustre-OST0000 complete [10697.698055] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10697.698755] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10697.712516] Lustre: Skipped 1 previous similar message [10697.732928] Lustre: Skipped 2 previous similar messages [10702.306691] Lustre: server umount lustre-MDT0000 complete [10705.764734] LustreError: 299089:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786016787 with bad export cookie 18025371392799265987 [10705.771166] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10705.776072] LustreError: 299089:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10706.211415] Lustre: server umount lustre-MDT0001 complete [10716.378042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10722.410721] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10728.222824] Lustre: server umount MGS complete [10742.270673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10742.708592] LustreError: 304887:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10742.732518] LustreError: 304887:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10742.760674] LustreError: 304900:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10742.861472] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10747.889403] LustreError: 304888:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10748.281656] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10753.005040] LustreError: 304887:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10757.976779] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10758.131875] LustreError: 304888:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10758.553576] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10764.167630] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10773.361989] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10779.716709] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10787.600213] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10788.146421] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10795.343562] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [10804.899809] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10848.737585] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10848.743513] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10848.778150] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10848.823109] Lustre: Skipped 1 previous similar message [10853.865975] LustreError: 306710:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10854.062936] Lustre: server umount lustre-OST0000 complete [10858.978217] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10858.990740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10858.996216] Lustre: Skipped 2 previous similar messages [10864.097813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10864.104593] Lustre: Skipped 1 previous similar message [10873.312171] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10873.882835] Lustre: server umount lustre-MDT0000 complete [10874.340178] LustreError: 304887:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10874.360609] LustreError: 304887:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10878.943781] LustreError: 304869:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786016960 with bad export cookie 18025371392799267814 [10878.955780] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10878.958752] LustreError: 304869:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10879.569971] Lustre: server umount lustre-MDT0001 complete [10893.176761] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [10896.122833] Key type lgssc unregistered [10896.479861] LNet: 308861:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10896.484880] LNetError: 308861:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10897.521793] LNet: Removed LNI 192.168.202.104@tcp [10898.564858] Key type .llcrypt unregistered [10898.567296] Key type ._llcrypt unregistered [10926.079501] Key type ._llcrypt registered [10926.084833] Key type .llcrypt registered [10926.327206] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [10944.770472] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [10946.213766] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10946.321499] alg: No test for adler32 (adler32-zlib) [10947.445785] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [10947.609119] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [10949.312185] Key type lgssc registered [10951.675625] Lustre: Echo OBD driver; http://www.lustre.org/ [10985.374709] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [11004.748828] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11004.844673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11006.175517] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11006.217932] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11006.325526] Lustre: lustre-MDT0000: new disk, initializing [11006.453225] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11006.477912] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11010.901796] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [11022.787113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11023.352610] Lustre: lustre-OST0000: new disk, initializing [11023.360480] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11023.365706] Lustre: Skipped 1 previous similar message [11023.375823] Lustre: 312919:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11023.476130] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11030.646216] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [11030.658444] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [11030.724672] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11031.100580] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [11967.465442] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11967.490479] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11968.670994] Lustre: server umount lustre-MDT0000 complete [11972.333818] LustreError: 312106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786018054 with bad export cookie 631576596069967112 [11972.336620] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11972.344318] LustreError: 312106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11972.462280] Lustre: server umount lustre-OST0000 complete [11996.853434] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [12007.008878] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [12043.893611] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12044.126863] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12044.158658] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12044.240800] Lustre: lustre-MDT0000: new disk, initializing [12044.295656] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12044.310589] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12048.609625] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12059.228273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12059.374177] Lustre: 317283:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12059.417972] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12059.421966] Lustre: Skipped 1 previous similar message [12059.561338] Lustre: lustre-MDT0001: new disk, initializing [12059.686115] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12059.725453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12059.738795] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12065.004267] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12075.224703] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12080.741960] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12087.125576] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12087.506645] Lustre: lustre-OST0000: new disk, initializing [12087.510773] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12087.518604] Lustre: 319250:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12087.615819] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12094.724871] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12097.158865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12097.183716] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12097.303959] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12103.583223] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12108.360262] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12108.700791] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12112.800694] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12112.979715] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12117.990248] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12117.991846] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12118.002468] LustreError: Skipped 1 previous similar message [12118.004744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12118.010889] Lustre: Skipped 1 previous similar message [12118.038469] Lustre: Skipped 1 previous similar message [12121.012118] Lustre: server umount lustre-OST0000 complete [12126.180364] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12126.186969] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12126.202911] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12128.229054] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12128.232648] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12131.381081] Lustre: server umount lustre-MDT0000 complete [12134.831596] LustreError: 317277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786018216 with bad export cookie 631576596072907931 [12134.842643] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12134.844350] LustreError: 317277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12135.191813] Lustre: server umount lustre-MDT0001 complete [12138.427641] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [12140.379539] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 08:10:19 (1786018219) [12147.198576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12147.565465] LustreError: 321565:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12147.573472] LustreError: 321565:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12147.613302] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12152.435769] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12152.802785] LustreError: 321566:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12157.925093] LustreError: 321565:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12162.567352] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12162.976762] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12167.597931] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12175.406871] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12180.791699] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12187.240172] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12187.579092] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12193.224385] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12200.540814] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12207.637745] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12207.912361] Lustre: lustre-OST0001: new disk, initializing [12207.916506] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12207.926922] Lustre: 324401:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12208.088898] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12213.364771] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12213.375744] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12213.416740] LustreError: 324614:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12214.449052] LustreError: 324614:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12214.460074] LustreError: 324614:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12215.524476] LustreError: 324614:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12215.545279] LustreError: 324614:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12216.195961] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12217.647410] LustreError: 324624:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12217.659956] LustreError: 324624:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [12221.731946] LustreError: 324614:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12221.743572] LustreError: 324614:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [12227.281502] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12227.551509] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12229.994835] LustreError: 324624:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12230.008674] LustreError: 324624:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [12232.721681] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff940b07cae000.ost_server_uuid 50 [12234.316278] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff940b07cae000.ost_server_uuid in FULL state after 0 sec [12239.707321] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff940b07cae000.ost_server_uuid 50 [12241.061754] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff940b07cae000.ost_server_uuid in FULL state after 0 sec [12242.364029] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12257.761715] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12257.772075] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12257.780184] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12257.788629] Lustre: Skipped 1 previous similar message [12257.805766] Lustre: Skipped 2 previous similar messages [12261.949748] Lustre: server umount lustre-OST0001 complete [12263.436609] LustreError: 323387:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.4@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12269.538753] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12269.548152] LustreError: Skipped 1 previous similar message [12269.551204] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12269.558146] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12269.565683] Lustre: Skipped 3 previous similar messages [12270.055363] LustreError: 323388:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12270.072890] LustreError: 323388:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12273.101645] Lustre: server umount lustre-OST0000 complete [12283.268376] Lustre: server umount lustre-MDT0000 complete [12285.416884] LustreError: 322569:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12286.675481] LustreError: 324412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786018368 with bad export cookie 631576596072908813 [12286.677712] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12286.682049] LustreError: 324412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [12287.183408] Lustre: server umount lustre-MDT0001 complete [12296.514908] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [12299.290813] Key type lgssc unregistered [12299.590063] LNet: 327090:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12299.599076] LNetError: 327090:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12299.614669] LNet: Removed LNI 192.168.202.104@tcp [12300.343614] Key type .llcrypt unregistered [12300.345747] Key type ._llcrypt unregistered [12323.519354] Key type ._llcrypt registered [12323.522268] Key type .llcrypt registered [12323.672328] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [12324.999420] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12325.183925] alg: No test for adler32 (adler32-zlib) [12326.159941] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [12326.372948] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [12328.040204] Key type lgssc registered [12329.162399] Lustre: Echo OBD driver; http://www.lustre.org/ [12337.059461] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12337.083376] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12338.475289] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12342.945266] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12348.903665] LustreError: 328510:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12348.916410] LustreError: 328510:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12350.229402] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12354.019239] LustreError: 328509:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12357.000164] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12357.501317] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12362.174339] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12372.741591] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12373.096124] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12380.366873] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12382.311469] LustreError: 330190:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12383.390762] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12388.837523] LustreError: 330666:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12388.869954] LustreError: 330666:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12388.937857] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12393.971174] LustreError: 330190:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12394.007364] LustreError: 330190:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12397.565739] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12397.895400] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12403.272137] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12407.482511] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12416.351499] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12424.048703] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12424.281171] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12526.050732] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12526.055044] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12526.058118] LustreError: Skipped 1 previous similar message [12526.065126] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12526.069455] Lustre: Skipped 1 previous similar message [12529.395981] Lustre: server umount lustre-OST0001 complete [12531.175519] LustreError: 331377:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12531.198052] LustreError: 331377:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12536.291863] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12536.314863] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12536.315109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12536.336892] Lustre: Skipped 2 previous similar messages [12540.875950] Lustre: server umount lustre-OST0000 complete [12546.535775] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12546.542227] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12546.551492] Lustre: Skipped 2 previous similar messages [12546.569590] Lustre: Skipped 1 previous similar message [12551.002437] Lustre: server umount lustre-MDT0000 complete [12551.651037] LustreError: 328504:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12551.684500] LustreError: 328504:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12554.490231] LustreError: 328491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786018636 with bad export cookie 749007107852569193 [12554.494072] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12554.504137] LustreError: 328491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12554.821706] Lustre: server umount lustre-MDT0001 complete [12564.940926] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [12567.890854] Key type lgssc unregistered [12568.199512] LNet: 333455:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12568.204389] LNetError: 333455:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12568.231588] LNet: Removed LNI 192.168.202.104@tcp [12569.291849] Key type .llcrypt unregistered [12569.297793] Key type ._llcrypt unregistered [12618.495745] Key type ._llcrypt registered [12618.498784] Key type .llcrypt registered [12618.630482] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [12619.897848] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12620.013430] alg: No test for adler32 (adler32-zlib) [12621.148835] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [12621.413583] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [12623.137311] Key type lgssc registered [12624.543944] Lustre: Echo OBD driver; http://www.lustre.org/ [12632.455111] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12632.480324] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12633.893829] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12638.147954] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12643.375948] LustreError: 335745:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12643.397905] LustreError: 335745:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12646.653588] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12647.074073] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12652.698840] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12655.726911] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12662.129562] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12662.500189] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12669.794677] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12671.741837] LustreError: 337234:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12677.099150] LustreError: 337671:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12678.129461] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12679.652659] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12679.847847] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12685.329272] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12687.473047] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12698.310863] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12708.724800] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12723.682371] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12723.691709] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12723.706154] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12726.246967] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12726.253534] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12726.269176] Lustre: Skipped 1 previous similar message [12726.285851] Lustre: Skipped 2 previous similar messages [12728.591115] Lustre: server umount lustre-MDT0000 complete [12731.364905] LustreError: 335739:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12731.376894] LustreError: 335739:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [12732.687097] LustreError: 335725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786018814 with bad export cookie 7969499735687998088 [12732.698199] LustreError: 335725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12732.698385] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12733.067698] Lustre: server umount lustre-MDT0001 complete [12747.920203] Lustre: server umount lustre-OST0000 complete [12761.730748] Lustre: server umount lustre-OST0001 complete [12781.717230] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [12794.708603] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12795.151041] LustreError: 341015:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12795.236742] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12799.254118] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12805.610170] LustreError: 341016:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12805.624602] LustreError: 341016:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12808.207201] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12808.859504] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12814.197707] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12817.870995] Lustre: 342155:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12824.484582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12830.530730] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12834.030613] LustreError: 342510:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12838.554836] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12838.825043] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12838.836109] Lustre: Skipped 1 previous similar message [12840.889221] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12840.919556] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12844.613559] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [12851.892493] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12856.370219] Lustre: 344026:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12861.413501] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12861.416538] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12861.424189] Lustre: Skipped 3 previous similar messages [12861.442806] Lustre: Skipped 5 previous similar messages [12866.536143] LustreError: 341016:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12866.556885] LustreError: 341016:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12866.575345] Lustre: server umount lustre-MDT0000 complete [12870.441975] LustreError: 340996:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786018952 with bad export cookie 7969499735687999789 [12870.450619] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12870.456265] LustreError: 340996:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12870.709450] Lustre: server umount lustre-MDT0001 complete [12884.730539] Lustre: server umount lustre-OST0000 complete [12887.521030] Lustre: 335007:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786018953/real 1786018953] req@ffff962b03565180 x1872776347633152/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786018969 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12887.577032] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12888.825556] Lustre: server umount lustre-OST0001 complete [12893.203729] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12895.335395] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12897.512922] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 08:22:56 (1786018976) [12918.181710] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [12927.610502] loop0: detected capacity change from 0 to 1099511627776 [12928.164602] SGI XFS with ACLs, security attributes, quota, no debug enabled [12928.176902] *** VALIDATE xfs *** [12928.207283] XFS (loop0): Mounting V5 Filesystem [12928.262144] XFS (loop0): Ending clean mount [12929.546643] loop1: detected capacity change from 0 to 18691697672192 [13047.113769] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [13058.095695] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [13098.929787] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13099.269651] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13099.317940] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13099.448684] Lustre: lustre-MDT0000: new disk, initializing [13099.547284] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13099.567431] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13105.692200] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13117.591207] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13117.696425] Lustre: 349160:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13117.795416] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13117.806781] Lustre: Skipped 1 previous similar message [13117.979213] Lustre: lustre-MDT0001: new disk, initializing [13118.259795] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13118.290166] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13123.711591] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13134.027161] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13139.096957] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13145.620779] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13145.929492] Lustre: lustre-OST0000: new disk, initializing [13145.934845] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13145.945777] Lustre: 351131:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13147.925535] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13147.945171] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13148.141523] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13152.097822] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13163.407935] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13169.087945] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13169.306398] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13174.398542] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13174.666342] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13178.850539] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13178.855075] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13178.871889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13178.913102] Lustre: Skipped 1 previous similar message [13183.973535] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13183.977751] Lustre: Skipped 2 previous similar messages [13191.136136] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13191.507961] Lustre: server umount lustre-OST0000 complete [13199.336901] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13199.347466] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13199.355352] Lustre: Skipped 1 previous similar message [13199.374398] Lustre: Skipped 3 previous similar messages [13201.929096] Lustre: server umount lustre-MDT0000 complete [13204.464153] LustreError: 350009:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13204.489033] LustreError: 350009:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13205.628275] LustreError: 349153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786019287 with bad export cookie 7969499735688001343 [13205.643496] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13206.103373] Lustre: server umount lustre-MDT0001 complete [13214.990832] XFS (loop0): Unmounting Filesystem [13220.547486] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 08:28:19 (1786019299) [13228.225348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13228.597246] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13228.607480] Lustre: Skipped 2 previous similar messages [13233.065746] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13241.656665] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13246.455730] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13254.554674] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13260.090765] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13267.879547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13274.886667] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13282.850753] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13297.122105] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13297.122233] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13297.136556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13297.154088] Lustre: Skipped 1 previous similar message [13301.511838] Lustre: server umount lustre-OST0000 complete [13311.535082] Lustre: server umount lustre-MDT0000 complete [13315.210121] LustreError: 353672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786019397 with bad export cookie 7969499735688002218 [13315.214210] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13315.223821] LustreError: 353672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [13315.591698] Lustre: server umount lustre-MDT0001 complete [13326.479988] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [13329.290716] Key type lgssc unregistered [13329.460865] LNet: 357324:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13329.465530] LNetError: 357324:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13329.471413] LNet: Removed LNI 192.168.202.104@tcp [13330.122174] Key type .llcrypt unregistered [13330.125202] Key type ._llcrypt unregistered [13361.361286] Key type ._llcrypt registered [13361.363594] Key type .llcrypt registered [13361.502564] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [13362.707135] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13362.779604] alg: No test for adler32 (adler32-zlib) [13364.108615] Lustre: Lustre: Build Version: 2.17.56_50_g837cd6c [13364.590329] LNet: Added LNI 192.168.202.104@tcp [8/256/0/180] [13366.272187] Key type lgssc registered [13368.098851] Lustre: Echo OBD driver; http://www.lustre.org/ [13377.624188] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13377.650158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13379.098523] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13383.801214] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13389.288426] LustreError: 358894:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13389.322156] LustreError: 358894:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13392.869399] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13393.319373] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13397.860785] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13406.490926] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13412.467545] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13418.794895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13418.942914] LustreError: 360717:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13419.204944] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13425.126314] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13432.396779] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13440.551217] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.4@tcp (stopping) [13444.580090] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13444.591593] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13444.609201] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13445.091732] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13445.642527] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.4@tcp (stopping) [13445.654469] Lustre: Skipped 2 previous similar messages [13446.673413] Lustre: server umount lustre-MDT0001 complete [13450.217739] LustreError: 358894:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13450.258592] LustreError: 358894:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13455.333585] LustreError: 359909:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13455.356138] LustreError: 359909:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13458.461402] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13458.535542] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13458.539485] Lustre: Found index 1 for lustre-MDT0001, updating log [13458.570386] Lustre: 358880:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13458.803186] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13464.053061] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13466.412444] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13468.655163] LustreError: 362464:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13468.658723] LustreError: 362464:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13468.666598] LustreError: 358885:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13468.748259] LustreError: 362468:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.104@tcp: cfg command failed: rc = -17 [13468.765412] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13468.765412] [13476.389905] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.4@tcp (stopping) [13479.407639] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13479.414746] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13480.765291] Lustre: server umount lustre-MDT0001 complete [13481.473077] LustreError: 361300:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.4@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13481.504986] LustreError: 361300:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13489.656530] LustreError: 359909:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13489.671892] LustreError: 359909:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [13493.549117] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13493.726181] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13493.735567] Lustre: Found index 1 for lustre-MDT0001, updating log [13493.805391] Lustre: 358880:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13494.092038] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13499.106344] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13499.385210] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13503.489058] LustreError: 363569:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13503.496624] LustreError: 363569:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13503.507987] LustreError: 358885:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13503.510855] LustreError: 358885:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13503.579955] LustreError: 363577:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13503.586224] LustreError: 363577:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13503.592617] LustreError: 363577:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.104@tcp: cfg command failed: rc = -17 [13503.604328] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.104@tcp [13503.604328] [13806.561962] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13806.568407] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13806.576424] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13806.580648] Lustre: Skipped 2 previous similar messages [13806.604262] Lustre: Skipped 3 previous similar messages [13810.620726] Lustre: server umount lustre-MDT0001 complete [13811.689025] LustreError: 358888:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13811.714689] LustreError: 358888:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13822.435580] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13822.773501] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13822.784652] Lustre: Found index 1 for lustre-MDT0001, updating log [13822.913173] Lustre: 358882:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13823.426696] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13828.635649] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13829.032468] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [13829.648541] LustreError: 371663:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13829.658947] LustreError: 371663:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13829.679950] LustreError: 358885:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13829.683047] LustreError: 358885:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13829.768518] LustreError: 371668:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13829.787406] LustreError: 371668:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.104@tcp: cfg command failed: rc = -114 [13829.807535] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13829.807535] [14151.137727] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14151.154823] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14151.155614] Lustre: Skipped 2 previous similar messages [14151.176414] Lustre: Skipped 3 previous similar messages [14157.822890] Lustre: server umount lustre-MDT0000 complete [14161.389746] LustreError: 358888:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14161.408368] LustreError: 358888:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [14161.868307] LustreError: 358874:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786020243 with bad export cookie 145815361407150432 [14161.873529] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14161.875757] LustreError: 358874:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [14166.501248] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14171.625284] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [14171.638034] Lustre: Skipped 4 previous similar messages [14176.224148] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [14176.490557] Lustre: server umount lustre-MDT0001 complete [14180.754501] Lustre: server umount lustre-OST0000 complete [14192.100628] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [14209.295151] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [14250.293651] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 08:45:29 (1786020329) [14257.095829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14257.456561] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14257.497389] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14257.625264] Lustre: lustre-MDT0000: new disk, initializing [14257.748977] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14257.786055] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14263.325225] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [14275.254393] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14275.519817] Lustre: 381946:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14275.623661] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14275.631498] Lustre: Skipped 1 previous similar message [14275.750815] Lustre: lustre-MDT0001: new disk, initializing [14275.900837] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14275.947683] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14275.965594] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14281.778788] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [14292.922735] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14299.361966] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14309.011828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14309.569504] Lustre: lustre-OST0000: new disk, initializing [14309.580915] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14309.596263] Lustre: 383914:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14309.748356] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14315.178771] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14315.192781] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14315.440126] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14319.966452] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_default_debug -1 all [14331.976938] Lustre: DEBUG MARKER: oleg204-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14341.090414] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14341.091381] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14341.108277] Lustre: Skipped 2 previous similar messages [14341.111098] Lustre: Skipped 2 previous similar messages [14342.294782] Lustre: server umount lustre-MDT0000 complete [14345.983128] LustreError: 381937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786020427 with bad export cookie 145815361407152385 [14345.992818] LustreError: 381937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [14345.997680] LustreError: 382751:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14345.998843] LustreError: MGC192.168.202.104@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14346.005767] LustreError: 382751:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000001dc63a24 x1872777126915584/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [14346.210890] LustreError: 381954:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14346.232636] LustreError: 381954:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [14346.397482] Lustre: server umount lustre-MDT0001 complete [14352.530808] Lustre: server umount lustre-OST0000 complete [14384.169852] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing set_hostid [14395.887361] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing load_modules_local [14429.176627] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14116 sec ========= 08:48:28 (1786020508) [14431.587757] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 08:48:30 (1786020510) === [14435.488143] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 08:48:34 (1786020514) === [14485.309908] Lustre: DEBUG MARKER: oleg204-server.virtnet: executing unload_modules_local [14488.673903] Key type lgssc unregistered [14488.999659] LNet: 390284:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14489.003326] LNetError: 390284:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14489.018641] LNet: Removed LNI 192.168.202.104@tcp [14490.103068] Key type .llcrypt unregistered [14490.105171] Key type ._llcrypt unregistered