[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 466268461 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524592K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.003081] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005014] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009012] pid_max: default: 32768 minimum: 301 [ 0.010125] LSM: Security Framework initializing [ 0.011055] Yama: becoming mindful. [ 0.012041] SELinux: Initializing. [ 0.013070] *** VALIDATE selinux *** [ 0.021410] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026473] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027153] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028110] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029134] *** VALIDATE tmpfs *** [ 0.030434] *** VALIDATE proc *** [ 0.031277] *** VALIDATE cgroup *** [ 0.032011] *** VALIDATE cgroup2 *** [ 0.033257] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034165] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036028] Spectre V2 : User space: Vulnerable [ 0.037030] Speculative Store Bypass: Vulnerable [ 0.040276] debug: unmapping init [mem 0xffffffff8f459000-0xffffffff8f460fff] [ 0.043183] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044655] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045021] ... version: 2 [ 0.046011] ... bit width: 48 [ 0.047010] ... generic registers: 4 [ 0.048011] ... value mask: 0000ffffffffffff [ 0.049013] ... max period: 00007fffffffffff [ 0.050014] ... fixed-purpose events: 3 [ 0.051014] ... event mask: 000000070000000f [ 0.053217] rcu: Hierarchical SRCU implementation. [ 0.055407] smp: Bringing up secondary CPUs ... [ 0.056448] x86: Booting SMP configuration: [ 0.057024] .... node #0, CPUs: #1 #2 #3 [ 0.060190] smp: Brought up 1 node, 4 CPUs [ 0.062014] smpboot: Max logical packages: 1 [ 0.063013] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.121335] node 0 deferred pages initialised in 55ms [ 0.125094] devtmpfs: initialized [ 0.126286] x86/mm: Memory block size: 128MB [ 0.129793] gcov: version magic: 0x41383552 [ 0.133129] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.134061] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.136330] pinctrl core: initialized pinctrl subsystem [ 0.138267] [ 0.138772] ************************************************************* [ 0.142015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.145012] ** ** [ 0.147010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.150014] ** ** [ 0.153013] ** This means that this kernel is built to expose internal ** [ 0.156013] ** IOMMU data structures, which may compromise security on ** [ 0.158014] ** your system. ** [ 0.161013] ** ** [ 0.164014] ** If you see this message and you are not debugging the ** [ 0.166021] ** kernel, report this immediately to your vendor! ** [ 0.168012] ** ** [ 0.171013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.173012] ************************************************************* [ 0.174378] NET: Registered protocol family 16 [ 0.177559] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.180089] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.183167] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.187052] cpuidle: using governor menu [ 0.189000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.191523] PCI: Using configuration type 1 for base access [ 0.193134] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.203072] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.206028] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.209091] cryptd: max_cpu_qlen set to 1000 [ 0.212235] ACPI: Added _OSI(Module Device) [ 0.214017] ACPI: Added _OSI(Processor Device) [ 0.215012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.217014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.222310] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.229223] ACPI: Interpreter enabled [ 0.232134] ACPI: PM: (supports S0 S3 S4 S5) [ 0.235030] ACPI: Using IOAPIC for interrupt routing [ 0.239317] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.247000] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.260758] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.263042] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.266021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.269076] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.274083] acpiphp: Slot [2] registered [ 0.275217] acpiphp: Slot [5] registered [ 0.277179] acpiphp: Slot [6] registered [ 0.279168] acpiphp: Slot [7] registered [ 0.280133] acpiphp: Slot [8] registered [ 0.282135] acpiphp: Slot [9] registered [ 0.283113] acpiphp: Slot [10] registered [ 0.285167] acpiphp: Slot [3] registered [ 0.286093] acpiphp: Slot [4] registered [ 0.288142] acpiphp: Slot [11] registered [ 0.290155] acpiphp: Slot [12] registered [ 0.291145] acpiphp: Slot [13] registered [ 0.294171] acpiphp: Slot [14] registered [ 0.296172] acpiphp: Slot [15] registered [ 0.297148] acpiphp: Slot [16] registered [ 0.299176] acpiphp: Slot [17] registered [ 0.301241] acpiphp: Slot [18] registered [ 0.304236] acpiphp: Slot [19] registered [ 0.308226] acpiphp: Slot [20] registered [ 0.312235] acpiphp: Slot [21] registered [ 0.315211] acpiphp: Slot [22] registered [ 0.319055] acpiphp: Slot [23] registered [ 0.321126] acpiphp: Slot [24] registered [ 0.322119] acpiphp: Slot [25] registered [ 0.324105] acpiphp: Slot [26] registered [ 0.326107] acpiphp: Slot [27] registered [ 0.327196] acpiphp: Slot [28] registered [ 0.328352] acpiphp: Slot [29] registered [ 0.330118] acpiphp: Slot [30] registered [ 0.331119] acpiphp: Slot [31] registered [ 0.333130] PCI host bridge to bus 0000:00 [ 0.334017] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.337025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.339025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.342026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.345024] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.348026] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.349153] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.353475] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.362651] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.381508] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.389022] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.392030] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.394026] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.396030] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.399557] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.402898] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.405041] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.409847] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.414015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.427021] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.434020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.441517] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.449023] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.458040] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.473013] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.484439] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.492012] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.502015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.522015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.531604] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.544028] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.554036] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.577026] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.589443] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.598016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.606020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.628015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.638411] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.647022] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.657024] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.676028] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.691217] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.699015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.707012] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.725016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.738519] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.743089] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.745509] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.748479] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.751273] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.756750] iommu: Default domain type: Passthrough [ 0.758432] SCSI subsystem initialized [ 0.759129] ACPI: bus type USB registered [ 0.761092] usbcore: registered new interface driver usbfs [ 0.763085] usbcore: registered new interface driver hub [ 0.765128] usbcore: registered new device driver usb [ 0.767189] pps_core: LinuxPPS API ver. 1 registered [ 0.769013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.772076] PTP clock support registered [ 0.775075] EDAC MC: Ver: 3.0.0 [ 0.777171] PCI: Using ACPI for IRQ routing [ 0.780109] NetLabel: Initializing [ 0.782025] NetLabel: domain hash size = 128 [ 0.783019] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.786110] NetLabel: unlabeled traffic allowed by default [ 0.788222] vgaarb: loaded [ 0.790006] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.792025] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.797475] clocksource: Switched to clocksource kvm-clock [ 0.917211] VFS: Disk quotas dquot_6.6.0 [ 0.919089] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.921887] *** VALIDATE ramfs *** [ 0.923137] *** VALIDATE hugetlbfs *** [ 0.924654] pnp: PnP ACPI init [ 0.926622] pnp: PnP ACPI: found 6 devices [ 0.944821] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.950107] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.952679] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.955256] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.958234] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.961142] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.964429] NET: Registered protocol family 2 [ 0.967128] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.972844] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.977051] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.982596] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.986072] TCP: Hash tables configured (established 65536 bind 65536) [ 0.989354] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.992441] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.995057] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.997573] NET: Registered protocol family 1 [ 1.001111] RPC: Registered named UNIX socket transport module. [ 1.003519] RPC: Registered udp transport module. [ 1.005556] RPC: Registered tcp transport module. [ 1.008946] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.011498] NET: Registered protocol family 44 [ 1.013680] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.015434] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.017324] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.020376] PCI: CLS 0 bytes, default 64 [ 1.022455] Unpacking initramfs... [ 2.444818] debug: unmapping init [mem 0xffff89d0fcc54000-0xffff89d0fffbffff] [ 2.449074] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.451834] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.455366] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.951589] Initialise system trusted keyrings [ 2.953354] Key type blacklist registered [ 2.955144] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.967726] zbud: loaded [ 2.971474] *** VALIDATE nfs *** [ 2.972791] *** VALIDATE nfs4 *** [ 2.974275] pstore: using deflate compression [ 2.977622] Platform Keyring initialized [ 3.086550] NET: Registered protocol family 38 [ 3.088387] Key type asymmetric registered [ 3.089973] Asymmetric key parser 'x509' registered [ 3.091864] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.095323] io scheduler mq-deadline registered [ 3.097238] io scheduler kyber registered [ 3.100509] io scheduler bfq registered [ 3.102781] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.106137] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.109417] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.112696] ACPI: Power Button [PWRF] [ 3.118216] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.125618] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.141931] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.149923] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.166392] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.195145] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.225594] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.230746] Non-volatile memory driver v1.3 [ 3.232647] Linux agpgart interface v0.103 [ 3.265928] virtio_blk virtio1: [vda] 149392 512-byte logical blocks (76.5 MB/72.9 MiB) [ 3.269719] vda: detected capacity change from 0 to 76488704 [ 3.285602] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.288406] vdb: detected capacity change from 0 to 1073741824 [ 3.305446] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.308762] vdc: detected capacity change from 0 to 2621440000 [ 3.328679] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.333398] vdd: detected capacity change from 0 to 2621440000 [ 3.353490] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.357018] vde: detected capacity change from 0 to 4294967296 [ 3.377379] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.380389] vdf: detected capacity change from 0 to 4294967296 [ 3.386600] libphy: Fixed MDIO Bus: probed [ 3.394703] usbcore: registered new interface driver usbserial_generic [ 3.397703] usbserial: USB Serial support registered for generic [ 3.401771] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.406699] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.408976] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.412909] mousedev: PS/2 mouse device common for all mice [ 3.416599] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.417518] rtc_cmos 00:05: RTC can wake from S4 [ 3.423576] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.425539] rtc_cmos 00:05: registered as rtc0 [ 3.428591] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.429874] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.435871] intel_pstate: CPU model not supported [ 3.439214] hid: raw HID events driver (C) Jiri Kosina [ 3.441168] usbcore: registered new interface driver usbhid [ 3.442853] usbhid: USB HID core driver [ 3.444204] drop_monitor: Initializing network drop monitor service [ 3.446311] Initializing XFRM netlink socket [ 3.448455] NET: Registered protocol family 10 [ 3.451088] Segment Routing with IPv6 [ 3.452996] NET: Registered protocol family 17 [ 3.456326] mpls_gso: MPLS GSO support [ 3.463144] RAS: Correctable Errors collector initialized. [ 3.465796] AVX version of gcm_enc/dec engaged. [ 3.467886] AES CTR mode by8 optimization enabled [ 3.556962] sched_clock: Marking stable (3556931492, 0)->(4491275351, -934343859) [ 3.561396] registered taskstats version 1 [ 3.563873] Loading compiled-in X.509 certificates [ 3.566402] zswap: loaded using pool lzo/zbud [ 3.593951] Key type big_key registered [ 3.607923] Key type encrypted registered [ 3.609759] ima: No TPM chip found, activating TPM-bypass! [ 3.612559] ima: Allocated hash algorithm: sha1 [ 3.614774] ima: No architecture policies found [ 3.616861] evm: Initialising EVM extended attributes: [ 3.619167] evm: security.selinux [ 3.620667] evm: security.ima [ 3.622054] evm: security.capability [ 3.623707] evm: HMAC attrs: 0x1 [ 3.626420] rtc_cmos 00:05: setting system clock to 2026-08-22 10:38:42 UTC (1787395122) [ 3.633741] debug: unmapping init [mem 0xffffffff90403000-0xffffffff905fffff] [ 3.636900] debug: unmapping init [mem 0xffffffff8f182000-0xffffffff8f458fff] [ 3.647159] Write protecting the kernel read-only data: 28672k [ 3.650688] debug: unmapping init [mem 0xffffffff8d803000-0xffffffff8d9fffff] [ 3.653853] debug: unmapping init [mem 0xffffffff8e114000-0xffffffff8e1fffff] [ 3.694655] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.702546] systemd[1]: Detected virtualization kvm. [ 3.704493] systemd[1]: Detected architecture x86-64. [ 3.706475] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.733505] systemd[1]: No hostname configured. [ 3.735241] systemd[1]: Set hostname to . [ 3.737217] random: systemd: uninitialized urandom read (16 bytes read) [ 3.739767] systemd[1]: Initializing machine ID from random generator. [ 3.788966] random: ln: uninitialized urandom read (6 bytes read) [ 3.883688] random: systemd: uninitialized urandom read (16 bytes read) [ 3.887320] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.892822] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.900772] systemd[1]: Reached target Paths. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Swap. [ OK ] Reached target Initrd Root Device. Starting Journal Service... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... Starting Create Volatile Files and Directories... [ OK ] Reached target Timers. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.624196] device-mapper: uevent: version 1.0.3 [ 4.627080] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.402522] virtio_net virtio0 ens2: renamed from eth0 [ 5.424066] random: fast init done [ 5.466517] scsi host0: ata_piix [ 5.482550] scsi host1: ata_piix [ 5.484394] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.487411] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.085290] dracut-initqueue[591]: RTNETLINK answers: File exists [ 10.129821] random: crng init done [ 10.131194] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.695915] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.939918] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.238252] SELinux: Disabled at runtime. [ 12.306917] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.317045] systemd[1]: Detected virtualization kvm. [ 12.319155] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.889058] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.892758] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.898183] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.902857] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.906523] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.918463] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.923486] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Huge Pages File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Stopped target Switch Root. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Local Encrypted Volumes. Starting Remount Root and Kernel File Systems... [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target Slices. Mounting Kernel Debug File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Created slice system-getty.slice. Starting Apply Kernel Variables... Activating swap /dev/disk/by-label/SWAP... [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ 13.125610] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 13.440128] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.765938] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.867725] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.969728] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.999153] EDAC sbridge: Ver: 1.1.2 [ 15.967426] Key type dns_resolver registered [ 16.268915] NFS: Registering the id_resolver key type [ 16.271458] Key type id_resolver registered [ 16.273211] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg336-server login: [ 43.659596] libcfs: loading out-of-tree module taints kernel. [ 43.680765] Key type ._llcrypt registered [ 43.682207] Key type .llcrypt registered [ 43.751054] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 53.224243] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 53.986681] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 53.996262] alg: No test for adler32 (adler32-zlib) [ 55.457507] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 56.634339] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 58.560172] Key type lgssc registered [ 60.827844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 80.483284] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 126.407598] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 137.823484] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 137.857178] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 139.172526] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 139.201525] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 139.296937] Lustre: lustre-MDT0000: new disk, initializing [ 139.416634] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 139.460175] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 143.208939] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 155.818880] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 155.925339] Lustre: 6487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 155.967388] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 155.972532] Lustre: Skipped 1 previous similar message [ 156.130766] Lustre: lustre-MDT0001: new disk, initializing [ 156.204082] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 156.227710] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 156.247303] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 160.593882] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 165.550684] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 175.630898] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 176.066667] Lustre: lustre-OST0000: new disk, initializing [ 176.070228] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 176.079756] Lustre: 8426:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 176.172838] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 177.663821] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 177.688809] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 177.791373] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 182.536366] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 196.696197] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 196.870778] Lustre: lustre-OST0001: new disk, initializing [ 196.873983] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 196.886261] Lustre: 9499:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 196.961473] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 203.090630] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 205.326058] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 205.336589] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 205.388522] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 214.232496] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 222.079471] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 228.355187] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing check_logdir /tmp/testlogs/ [ 234.111420] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing yml_node [ 238.624404] Lustre: DEBUG MARKER: Client: 2.17.54.108 [ 240.933956] Lustre: DEBUG MARKER: MDS: 2.17.54.108 [ 243.203471] Lustre: DEBUG MARKER: OSS: 2.17.54.108 [ 244.686436] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Aug 22 06:42:41 EDT 2026 [ 261.376170] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 262.778383] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 282.088201] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 282.094693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 282.116761] Lustre: Skipped 3 previous similar messages [ 282.121241] Lustre: Skipped 2 previous similar messages [ 285.998372] Lustre: server umount lustre-MDT0000 complete [ 291.824844] LustreError: 8425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395410 with bad export cookie 16458323107769884627 [ 291.835354] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 291.837323] LustreError: 8425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 292.327830] LustreError: 12439:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 292.349406] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 292.365380] LustreError: 12439:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 292.398169] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 292.410106] Lustre: Skipped 1 previous similar message [ 296.363134] LustreError: 8434:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 296.363266] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 296.389360] LustreError: 8434:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 296.392826] Lustre: Skipped 1 previous similar message [ 298.426882] Lustre: server umount lustre-MDT0001 complete [ 302.290591] Lustre: server umount lustre-OST0000 complete [ 302.654184] hrtimer: interrupt took 4177350 ns [ 306.302713] Lustre: server umount lustre-OST0001 complete [ 313.474649] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 323.497731] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 362.854851] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 363.115722] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 363.144028] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 363.218176] Lustre: lustre-MDT0000: new disk, initializing [ 363.314748] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 363.348250] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 368.157558] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 380.004086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 380.164063] Lustre: 15051:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 380.273385] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 380.279683] Lustre: Skipped 1 previous similar message [ 380.420458] Lustre: lustre-MDT0001: new disk, initializing [ 380.620492] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 380.691206] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 380.708467] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 385.716543] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 395.840756] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 401.649900] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 408.423231] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 408.796697] Lustre: lustre-OST0000: new disk, initializing [ 408.805766] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 408.824177] Lustre: 17020:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 408.968061] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 410.529185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 410.553602] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 410.674267] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 415.860118] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 426.052789] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 430.965269] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 431.271917] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 436.299854] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 436.475413] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 441.321723] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 441.327246] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 441.327377] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 441.347407] Lustre: Skipped 2 previous similar messages [ 444.758618] Lustre: server umount lustre-OST0000 complete [ 451.554970] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 451.566255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 451.579056] Lustre: Skipped 1 previous similar message [ 451.591542] Lustre: Skipped 2 previous similar messages [ 454.840246] LustreError: 18558:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x736d61726170:0x3:0x0].0x0 (ffff89d146faf500) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 456.675932] LustreError: 15921:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 456.692898] LustreError: 15921:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 456.921836] Lustre: server umount lustre-MDT0000 complete [ 461.033065] LustreError: 15044:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395579 with bad export cookie 16458323107769886153 [ 461.035619] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 461.043494] LustreError: 15044:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 461.359750] Lustre: server umount lustre-MDT0001 complete [ 466.719821] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 06:46:23 (1787395583) [ 473.883955] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 474.244762] LustreError: 19287:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 474.303488] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 478.344867] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 479.714518] LustreError: 19288:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 486.854386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 491.971657] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 498.763690] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 504.105685] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 510.374153] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 516.399207] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 522.933431] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 526.075853] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 533.481822] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 533.492059] LustreError: Skipped 1 previous similar message [ 533.499387] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 533.514352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 538.131777] Lustre: server umount lustre-MDT0000 complete [ 540.643960] LustreError: 20291:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 540.655278] LustreError: 20291:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 541.259259] LustreError: 19268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395660 with bad export cookie 16458323107769887042 [ 541.260237] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 541.274770] LustreError: 19268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 541.532613] Lustre: server umount lustre-MDT0001 complete [ 555.218681] Lustre: server umount lustre-OST0000 complete [ 577.702622] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 589.064027] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 589.725339] LustreError: 23667:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 589.813900] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 589.819693] Lustre: Skipped 2 previous similar messages [ 594.594597] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 603.168456] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 607.762433] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 611.430885] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 618.756925] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 624.669576] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 633.250523] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 633.380667] Lustre: lustre-OST0001: new disk, initializing [ 633.384619] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 633.390187] Lustre: 25981:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 635.505802] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 635.516239] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 635.641862] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 639.624066] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 648.649673] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 651.964643] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 665.057350] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 665.067121] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 665.081965] Lustre: Skipped 2 previous similar messages [ 665.093969] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 665.096725] Lustre: Skipped 3 previous similar messages [ 669.705783] Lustre: server umount lustre-MDT0000 complete [ 671.722878] LustreError: 23663:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 671.742535] LustreError: 23663:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 673.207979] LustreError: 26746:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395792 with bad export cookie 16458323107769887889 [ 673.210593] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 673.222031] LustreError: 26746:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 673.519257] Lustre: server umount lustre-MDT0001 complete [ 686.436958] Lustre: server umount lustre-OST0000 complete [ 701.181253] Lustre: server umount lustre-OST0001 complete [ 719.649386] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 729.219808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 729.613628] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 729.618939] Lustre: Skipped 3 previous similar messages [ 733.669530] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 739.812090] LustreError: 29005:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 739.831693] LustreError: 29005:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 742.388249] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 747.282868] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 750.036781] Lustre: 30151:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 756.432356] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 762.490674] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 770.880577] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 777.531506] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 784.475939] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 787.170827] Lustre: 32028:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 794.093304] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 794.105273] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 794.122633] Lustre: Skipped 3 previous similar messages [ 794.128524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 794.131575] Lustre: Skipped 4 previous similar messages [ 799.254639] Lustre: server umount lustre-MDT0000 complete [ 802.833769] LustreError: 28986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395921 with bad export cookie 16458323107769889184 [ 802.839115] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 802.855343] LustreError: 28986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 803.127747] Lustre: server umount lustre-MDT0001 complete [ 817.013193] Lustre: server umount lustre-OST0000 complete [ 831.012905] Lustre: server umount lustre-OST0001 complete [ 838.278130] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 06:52:35 (1787395955) [ 866.644398] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 876.560989] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 876.907147] LustreError: 34830:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 876.939333] LustreError: 34830:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 881.631520] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 890.663174] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 896.068213] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 899.093837] Lustre: 35978:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 905.446995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 912.315316] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 921.280156] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 927.682629] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 935.206696] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 939.049556] Lustre: 37857:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 947.171854] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 947.184345] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 947.188765] Lustre: Skipped 5 previous similar messages [ 947.195240] Lustre: Skipped 5 previous similar messages [ 950.353808] Lustre: server umount lustre-MDT0000 complete [ 953.717037] LustreError: 34810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396072 with bad export cookie 16458323107769890290 [ 953.722171] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 953.735677] LustreError: 34810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 954.084672] Lustre: server umount lustre-MDT0001 complete [ 968.597827] Lustre: server umount lustre-OST0000 complete [ 981.563793] Lustre: server umount lustre-OST0001 complete [ 989.863329] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 06:55:06 (1787396106) [ 1007.088652] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 1016.750465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1017.112622] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1017.117412] Lustre: Skipped 7 previous similar messages [ 1020.879225] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1028.220074] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1032.726949] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1035.433473] Lustre: 41158:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1042.028528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1048.467418] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1055.622527] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1061.413145] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1067.533678] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1070.656806] Lustre: 43036:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1071.583951] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1071.589361] Lustre: Skipped 1 previous similar message [ 1074.658184] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1074.670239] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1077.735039] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1077.749385] Lustre: Skipped 6 previous similar messages [ 1080.343326] Lustre: server umount lustre-MDT0000 complete [ 1083.017204] LustreError: 41162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396201 with bad export cookie 16458323107769891396 [ 1083.022052] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1083.029489] LustreError: 41162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1089.326607] Lustre: server umount lustre-MDT0001 complete [ 1096.124961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1107.430679] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xe467b5aaa53a3eab [ 1107.440219] Lustre: MGC192.168.203.136@tcp: Connection restored to 0@lo (at 0@lo) [ 1111.335722] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1118.013893] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1119.109376] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1124.116145] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1124.857134] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1124.866246] Lustre: Skipped 1 previous similar message [ 1130.865166] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1132.272852] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1139.557958] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1142.603899] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1150.450187] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1154.837725] Lustre: server umount lustre-MDT0000 complete [ 1155.568839] LustreError: 43909:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1155.590460] LustreError: 43909:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 34 previous similar messages [ 1158.704396] LustreError: 41162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396277 with bad export cookie 16458323107769892523 [ 1158.706261] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1158.712724] LustreError: 41162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1159.019835] Lustre: server umount lustre-MDT0001 complete [ 1174.028511] Lustre: server umount lustre-OST0000 complete [ 1177.056178] Lustre: 3623:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787396279/real 1787396279] req@ffff89d145acc000 x1874219683441024/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787396295 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1177.767904] Lustre: server umount lustre-OST0001 complete [ 1186.894787] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 06:58:23 (1787396303) [ 1201.409290] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 1210.651913] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1215.174075] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1222.973215] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1227.557670] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1230.313173] Lustre: 48792:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1236.040276] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1241.499837] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1248.944253] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1255.321133] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1262.313267] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1266.989782] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1279.459020] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1279.479773] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1279.508054] Lustre: Skipped 10 previous similar messages [ 1293.280157] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1293.664192] Lustre: server umount lustre-MDT0000 complete [ 1297.893688] LustreError: 48397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396416 with bad export cookie 16458323107769893314 [ 1297.901497] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1297.901670] LustreError: 48397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1298.312469] Lustre: server umount lustre-MDT0001 complete [ 1312.731400] Lustre: server umount lustre-OST0000 complete [ 1326.034538] Lustre: server umount lustre-OST0001 complete [ 1333.447726] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 07:00:50 (1787396450) [ 1335.267709] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1336.907675] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 07:00:53 (1787396453) [ 1346.305149] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 1353.542642] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 1373.604044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1373.862434] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1373.952815] Lustre: lustre-MDT0000: new disk, initializing [ 1374.066075] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1377.768358] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1386.493539] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1392.529075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1392.766725] Lustre: lustre-OST0000: new disk, initializing [ 1392.773533] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1392.777619] Lustre: Skipped 1 previous similar message [ 1392.781816] Lustre: 54882:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1394.267376] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1394.283648] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1394.321384] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1398.369742] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1407.784536] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1410.428880] Lustre: DEBUG MARKER: create test files [ 1440.225486] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1440.238544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1440.247352] Lustre: Skipped 24 previous similar messages [ 1444.793434] Lustre: server umount lustre-OST0000 complete [ 1448.342788] Lustre: server umount lustre-MDT0000 complete [ 1457.532782] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 1460.052333] Key type lgssc unregistered [ 1460.381652] LNet: 56509:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1460.393548] LNetError: 56509:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1460.403763] LNet: Removed LNI 192.168.203.136@tcp [ 1461.195404] Key type .llcrypt unregistered [ 1461.198251] Key type ._llcrypt unregistered [ 1491.631537] Key type ._llcrypt registered [ 1491.637178] Key type .llcrypt registered [ 1491.785801] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 1493.137940] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1493.151212] alg: No test for adler32 (adler32-zlib) [ 1494.254794] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 1494.578345] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 1496.216202] Key type lgssc registered [ 1497.195937] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1504.619696] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1504.645491] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1506.026359] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1509.435291] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1515.705833] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1521.379264] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1521.633538] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1526.328389] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1530.866501] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1530.952362] Lustre: 59243:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1530.958486] Lustre: 59243:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1530.964811] Lustre: 59243:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1530.970255] Lustre: 59243:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1530.976568] Lustre: 59243:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1530.991533] Lustre: 59243:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1533.085832] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1538.168971] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1543.015852] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1551.335613] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1551.344756] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1551.369660] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1551.544260] Lustre: server umount lustre-OST0000 complete [ 1554.980814] Lustre: server umount lustre-MDT0000 complete [ 1564.450236] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 1566.947818] Key type lgssc unregistered [ 1567.185727] LNet: 60848:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1567.194238] LNetError: 60848:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1567.209956] LNet: Removed LNI 192.168.203.136@tcp [ 1567.958274] Key type .llcrypt unregistered [ 1567.965475] Key type ._llcrypt unregistered [ 1591.959919] Key type ._llcrypt registered [ 1591.966973] Key type .llcrypt registered [ 1592.105056] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 1593.009589] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1593.049882] alg: No test for adler32 (adler32-zlib) [ 1594.108276] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 1594.299200] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 1595.944327] Key type lgssc registered [ 1596.833160] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1602.557622] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1602.577824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1603.931710] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1607.265104] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1612.749619] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1618.042623] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1618.305176] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1623.200645] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1628.589457] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1628.660045] Lustre: 63662:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1628.669793] Lustre: 63662:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1628.677765] Lustre: 63662:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1628.682482] Lustre: 63662:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1628.689366] Lustre: 63662:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1628.694677] Lustre: 63662:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1628.928887] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1633.766620] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1644.001294] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1644.006291] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1644.028801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1646.172422] Lustre: server umount lustre-OST0000 complete [ 1648.982929] Lustre: server umount lustre-MDT0000 complete [ 1656.233627] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 1658.109322] Key type lgssc unregistered [ 1658.354614] LNet: 65219:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1658.363240] LNetError: 65219:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1658.385587] LNet: Removed LNI 192.168.203.136@tcp [ 1659.002840] Key type .llcrypt unregistered [ 1659.005147] Key type ._llcrypt unregistered [ 1673.099797] Key type ._llcrypt registered [ 1673.103899] Key type .llcrypt registered [ 1673.177958] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 1686.218099] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 1687.121751] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1687.176893] alg: No test for adler32 (adler32-zlib) [ 1688.180445] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 1688.344891] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 1689.992578] Key type lgssc registered [ 1690.872647] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1718.810502] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1718.828576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1720.032784] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1720.080238] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1720.164641] Lustre: lustre-MDT0000: new disk, initializing [ 1720.223102] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1720.240613] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1724.602320] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1733.465639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1733.616754] Lustre: 68350:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1733.668525] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1733.676296] Lustre: Skipped 1 previous similar message [ 1733.752397] Lustre: lustre-MDT0001: new disk, initializing [ 1733.831805] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1733.867186] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1733.884234] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1737.533651] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1744.903470] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1749.071104] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1754.551774] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1754.798726] Lustre: lustre-OST0000: new disk, initializing [ 1754.805781] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1754.812829] Lustre: 70318:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1754.891785] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1760.657488] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1762.367400] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1762.379663] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1762.515253] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1769.340884] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1774.140473] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1774.340348] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1779.217421] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1779.468681] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1783.270988] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1783.274037] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1783.284970] LustreError: Skipped 1 previous similar message [ 1783.288079] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1783.300492] Lustre: Skipped 1 previous similar message [ 1783.322453] Lustre: Skipped 1 previous similar message [ 1787.550241] Lustre: server umount lustre-OST0000 complete [ 1793.513530] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1793.543767] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1793.554186] Lustre: Skipped 1 previous similar message [ 1797.581394] Lustre: server umount lustre-MDT0000 complete [ 1800.925371] LustreError: 68342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396919 with bad export cookie 13781866200482833517 [ 1800.931277] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1800.936535] LustreError: 68342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1801.230621] Lustre: server umount lustre-MDT0001 complete [ 1808.404521] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 07:08:45 (1787396925) [ 1817.452887] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 1850.436988] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 1857.408989] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 1885.943089] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1886.113991] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1886.148872] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1886.262512] Lustre: lustre-MDT0000: new disk, initializing [ 1886.364406] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1886.378313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1890.030294] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1899.426240] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1899.589900] Lustre: 76342:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1899.668525] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1899.672865] Lustre: Skipped 1 previous similar message [ 1899.781984] Lustre: lustre-MDT0001: new disk, initializing [ 1899.894653] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1899.927825] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1899.942223] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1904.134783] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1914.014366] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1919.760159] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1926.097410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1926.440230] Lustre: lustre-OST0000: new disk, initializing [ 1926.453915] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1926.460147] Lustre: 78311:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1926.561598] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1931.816617] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1931.836467] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1931.959309] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1932.115038] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1940.717676] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1945.372478] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1945.550083] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1949.850245] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1950.032170] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1952.737136] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1952.742284] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1952.744045] LustreError: Skipped 1 previous similar message [ 1952.745920] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1952.780148] Lustre: Skipped 2 previous similar messages [ 1957.870843] Lustre: server umount lustre-OST0000 complete [ 1961.441361] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1961.447400] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1961.472968] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1961.477372] Lustre: Skipped 1 previous similar message [ 1968.100364] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1968.109138] Lustre: Skipped 3 previous similar messages [ 1975.264254] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1975.416511] Lustre: server umount lustre-MDT0000 complete [ 1977.320201] LustreError: 76355:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1977.343199] LustreError: 76355:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 1978.410780] LustreError: 76334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397097 with bad export cookie 13781866200482834399 [ 1978.424078] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1978.428889] LustreError: 76334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1978.761626] Lustre: server umount lustre-MDT0001 complete [ 1986.909535] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 07:11:43 (1787397103) [ 1993.543258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1994.010496] LustreError: 80723:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1994.142967] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1998.140199] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 1999.339541] LustreError: 80724:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2003.428888] LustreError: 80723:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2005.003546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2009.646667] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2017.273674] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2023.517822] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2030.229608] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2030.440933] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2030.448532] Lustre: Skipped 1 previous similar message [ 2037.799451] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2046.647462] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2050.266138] Lustre: 83392:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 3c398eae-859a-4500-a678-d6b94d939fac at adminstrative request [ 2054.134142] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2054.152395] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2064.358394] Lustre: MGS: Client 3c398eae-859a-4500-a678-d6b94d939fac (at 0@lo) reconnecting [ 2064.385249] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xbf43064a35bdeb51 to 0xbf43064a35bdedf1 [ 2064.403665] Lustre: MGC192.168.203.136@tcp: Connection restored to 0@lo (at 0@lo) [ 2088.780305] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2089.093901] Lustre: lustre-OST0001: new disk, initializing [ 2089.111744] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2089.122326] Lustre: 83785:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2096.263628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2096.292341] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2096.452887] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2096.638459] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2105.671757] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2111.977522] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2111.986793] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2111.988881] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2111.997375] Lustre: Skipped 2 previous similar messages [ 2112.011880] Lustre: Skipped 3 previous similar messages [ 2123.232215] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2123.385668] Lustre: server umount lustre-OST0001 complete [ 2127.330160] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2127.331607] LustreError: 82543:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2127.352280] Lustre: Skipped 1 previous similar message [ 2127.378762] LustreError: 82543:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2132.454668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2132.467164] Lustre: Skipped 7 previous similar messages [ 2133.404437] Lustre: server umount lustre-OST0000 complete [ 2142.903374] Lustre: server umount lustre-MDT0000 complete [ 2147.350857] LustreError: 83784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397266 with bad export cookie 13781866200482835953 [ 2147.367088] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2147.374619] LustreError: 83784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2147.711756] Lustre: server umount lustre-MDT0001 complete [ 2155.976686] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 07:14:32 (1787397272) [ 2157.197556] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2158.563683] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 07:14:35 (1787397275) [ 2160.322625] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2162.245892] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 07:14:38 (1787397278) [ 2163.721701] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2165.509346] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 07:14:42 (1787397282) [ 2167.093442] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2168.817870] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 07:14:45 (1787397285) [ 2170.369812] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2175.913571] loop: module loaded [ 2175.936898] loop0: detected capacity change from 0 to 4294967296 [ 2176.013249] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2177.385659] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2180.413154] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2183.903977] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2209.369288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2209.416723] LustreError: 87908:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2209.423763] LustreError: 87908:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2209.429128] LustreError: 87908:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2209.438384] LustreError: 87908:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2233.467645] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 2244.214910] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 2281.239080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2281.496553] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2281.525089] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2281.613183] Lustre: lustre-MDT0000: new disk, initializing [ 2281.679100] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2281.685186] Lustre: Skipped 1 previous similar message [ 2281.698419] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2286.613763] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2297.032827] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2297.168964] Lustre: 90775:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2297.413483] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2301.740096] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2310.462485] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2316.477710] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2324.512210] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2324.831406] Lustre: 92747:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2331.279068] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2331.469705] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2332.256174] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2341.842758] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2346.861211] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2347.136866] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2351.518712] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2351.769435] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2357.217833] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2357.235668] LustreError: Skipped 1 previous similar message [ 2357.239373] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2357.257686] Lustre: Skipped 2 previous similar messages [ 2357.263063] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2357.271720] Lustre: Skipped 5 previous similar messages [ 2359.792863] Lustre: server umount lustre-OST0000 complete [ 2378.208154] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2378.473612] Lustre: server umount lustre-MDT0000 complete [ 2382.047403] LustreError: 91572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397500 with bad export cookie 13781866200482836569 [ 2382.056398] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2382.068950] LustreError: 91572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2382.615631] Lustre: server umount lustre-MDT0001 complete [ 2391.038747] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 07:18:27 (1787397507) [ 2397.699549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2398.160476] LustreError: 95158:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2398.187125] LustreError: 95158:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2403.029795] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2411.274096] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2411.830025] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2411.839145] Lustre: Skipped 3 previous similar messages [ 2416.052541] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2423.430593] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2430.028450] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2436.773883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2442.834201] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2449.775307] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2455.674767] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2456.023226] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2462.103518] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2462.481325] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2471.532913] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2471.893762] Lustre: lustre-OST0001: new disk, initializing [ 2471.898568] Lustre: Skipped 2 previous similar messages [ 2471.906313] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2471.921601] Lustre: Skipped 3 previous similar messages [ 2471.931234] Lustre: 98528:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2478.674380] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2481.856408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2481.865549] Lustre: Skipped 2 previous similar messages [ 2481.879267] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2482.030235] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2487.556033] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2491.716486] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2492.005076] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2496.853313] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2497.255396] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2519.357823] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2542.006168] Lustre: Failing over lustre-MDT0000 [ 2542.601819] Lustre: server umount lustre-MDT0000 complete [ 2543.596737] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2543.599489] LustreError: 95158:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2543.605399] Lustre: Skipped 6 previous similar messages [ 2543.620924] LustreError: 95158:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2559.968336] Lustre: 66574:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787397662/real 1787397662] req@00000000e17e8916 x1874221394571264/t0(0) o400->MGC192.168.203.136@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787397678 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2560.002116] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2565.860834] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2565.864520] LDISKFS-fs (dm-0): recovery complete [ 2565.872485] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2569.190930] LustreError: 66573:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000000dc3703a x1874221394579072/t0(0) o250->MGC192.168.203.136@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2569.682174] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2571.298596] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2571.302602] Lustre: 100242:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2574.823963] Lustre: 95887:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2574.847695] Lustre: 95887:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 2574.877292] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2575.024241] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2575.257123] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2575.361505] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2575.692214] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2575.709151] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2576.037585] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2576.047832] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2576.067496] Lustre: 101376:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2576.086718] Lustre: 101376:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 2576.744519] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2576.758785] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2577.072089] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2577.093523] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2578.107231] Lustre: 101376:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2578.125122] Lustre: 101376:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 5 previous similar messages [ 2578.755492] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2578.765667] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 5 previous similar messages [ 2579.088214] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2579.095736] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 5 previous similar messages [ 2582.312466] Lustre: 101376:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2582.329745] Lustre: 101376:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 2582.979819] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2582.999258] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2583.320349] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2583.331048] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2590.621158] Lustre: 101376:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2590.650715] Lustre: 101376:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 25 previous similar messages [ 2590.988623] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2591.009831] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2591.358433] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2591.365851] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2606.833938] Lustre: 101376:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2606.850462] Lustre: 101376:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 50 previous similar messages [ 2607.184492] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2607.194023] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2607.512180] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2607.529522] LustreError: 101376:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2631.185284] Lustre: 101376:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2631.196651] Lustre: 101376:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2631.207176] Lustre: 101376:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 5cc1f889-208d-4148-85c1-c85d23d61ea6@192.168.203.36@tcp [ 2631.223587] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2631.229059] Lustre: 101376:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@000000009e8f84df x1874221379687040/t0(8589934773) o101->5cc1f889-208d-4148-85c1-c85d23d61ea6@192.168.203.36@tcp:190/0 lens 592/0 e 0 to 0 dl 1787397760 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2631.255752] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2631.278062] Lustre: lustre-MDT0000: Denying connection for new client 5cc1f889-208d-4148-85c1-c85d23d61ea6 (at 192.168.203.36@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2631.284663] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2631.332784] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2631.338447] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2631.339958] Lustre: Skipped 2 previous similar messages [ 2631.372762] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2631.374047] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2645.988766] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2646.000991] LustreError: Skipped 2 previous similar messages [ 2646.010429] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2646.021943] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2646.026270] Lustre: Skipped 7 previous similar messages [ 2651.847144] Lustre: server umount lustre-OST0000 complete [ 2656.738642] LustreError: 96980:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2656.762510] LustreError: 96980:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 36 previous similar messages [ 2662.208884] Lustre: server umount lustre-OST0001 complete [ 2672.017419] Lustre: server umount lustre-MDT0000 complete [ 2675.647058] LustreError: 95140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397794 with bad export cookie 13781866200482887781 [ 2675.648642] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2675.658191] LustreError: 95140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2676.019489] Lustre: server umount lustre-MDT0001 complete [ 2684.615247] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 07:23:21 (1787397801) [ 2691.049685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2691.058716] Lustre: *** cfs_fail_loc=197, val=0*** [ 2706.657457] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2711.776566] LustreError: 103356:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.136@tcp: failed processing log, type 4: rc = -110 [ 2716.907695] Lustre: *** cfs_fail_loc=197, val=0*** [ 2716.913405] Lustre: Skipped 65 previous similar messages [ 2737.505682] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2737.516907] Lustre: Skipped 3 previous similar messages [ 2743.782479] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2750.760570] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2752.486245] Lustre: server umount lustre-OST0000 complete [ 2760.268459] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 07:24:36 (1787397876) [ 2780.468744] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2780.900794] LustreError: 104976:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2780.919740] LustreError: 104976:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2784.784712] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2793.351250] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2798.547972] LustreError: 105007:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2798.566266] LustreError: 105007:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2798.573364] LustreError: 105007:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 2799.226051] Lustre: server umount lustre-MDT0000 complete [ 2833.791096] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 2842.740367] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 2876.177420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2876.422596] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2876.442819] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2876.540666] Lustre: lustre-MDT0000: new disk, initializing [ 2876.638238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2880.807767] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2890.471869] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2890.596481] Lustre: 109180:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2890.851079] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2894.992205] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2903.347820] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2908.373029] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2914.862438] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2915.190441] Lustre: 111150:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2921.461948] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 2923.166396] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2930.141954] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2935.432332] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2935.643597] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2940.683475] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2940.982781] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2943.974832] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2943.977473] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2944.002454] LustreError: Skipped 1 previous similar message [ 2944.028135] Lustre: Skipped 5 previous similar messages [ 2944.043634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2944.052026] Lustre: Skipped 11 previous similar messages [ 2949.669426] Lustre: server umount lustre-OST0000 complete [ 2968.032143] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2968.230174] Lustre: server umount lustre-MDT0000 complete [ 2969.569659] LustreError: 109191:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2969.591752] LustreError: 109191:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2972.059955] LustreError: 111148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398090 with bad export cookie 13781866200482898400 [ 2972.068047] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2972.070383] LustreError: 111148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2972.374989] Lustre: server umount lustre-MDT0001 complete [ 2979.612825] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 07:28:16 (1787398096) [ 2988.641978] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 2991.548518] Key type lgssc unregistered [ 2991.881202] LNet: 113638:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2991.889336] LNetError: 113638:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2991.905257] LNet: Removed LNI 192.168.203.136@tcp [ 2992.759744] Key type .llcrypt unregistered [ 2992.762366] Key type ._llcrypt unregistered [ 3012.075311] Key type ._llcrypt registered [ 3012.077921] Key type .llcrypt registered [ 3012.235967] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 3028.682305] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 3029.945489] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3029.959598] alg: No test for adler32 (adler32-zlib) [ 3031.267616] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 3031.611306] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 3033.272820] Key type lgssc registered [ 3034.569832] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3073.787406] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3073.837382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3075.248338] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3075.283381] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3075.426144] Lustre: lustre-MDT0000: new disk, initializing [ 3075.495514] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3075.516031] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3080.015455] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3089.945814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3090.259351] Lustre: lustre-OST0000: new disk, initializing [ 3090.274046] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3090.282893] Lustre: Skipped 1 previous similar message [ 3090.288975] Lustre: 117812:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3090.393129] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3096.155748] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3098.733814] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3098.746906] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3098.784713] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3106.106711] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3106.345596] Lustre: lustre-OST0001: new disk, initializing [ 3106.358762] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3106.371169] Lustre: 118677:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3106.446783] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3112.111844] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3116.130359] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3116.140797] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3116.172362] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3121.122812] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3121.141881] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3126.243021] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3126.253807] Lustre: Skipped 2 previous similar messages [ 3126.784954] Lustre: server umount lustre-MDT0000 complete [ 3133.446433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3138.357171] LustreError: 116998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398257 with bad export cookie 17187638145356993337 [ 3138.373640] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3138.424488] Lustre: server umount lustre-OST0000 complete [ 3142.132992] Lustre: server umount lustre-OST0001 complete [ 3150.757593] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 07:31:07 (1787398267) [ 3172.538836] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 07:31:29 (1787398289) [ 3181.718750] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 3201.944561] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 07:31:58 (1787398318) [ 3241.390655] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 3252.411556] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 3287.859280] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3288.128532] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3288.166340] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3288.244773] Lustre: lustre-MDT0000: new disk, initializing [ 3288.321827] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3288.337683] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3292.563047] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3303.089440] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3303.247225] Lustre: 126428:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3303.312392] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3303.320482] Lustre: Skipped 1 previous similar message [ 3303.413673] Lustre: lustre-MDT0001: new disk, initializing [ 3303.512677] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3303.551150] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3303.566178] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3308.205871] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3317.937897] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3323.683840] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3331.475450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3331.964444] Lustre: lustre-OST0000: new disk, initializing [ 3331.967240] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3331.987808] Lustre: 128396:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3332.157650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3338.931337] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3339.886377] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3339.919451] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3340.069888] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3349.278488] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3353.958331] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3354.208458] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3358.970178] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3359.252365] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3365.857891] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3365.873121] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3365.892119] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3365.906143] Lustre: Skipped 2 previous similar messages [ 3367.791736] Lustre: server umount lustre-OST0000 complete [ 3375.088127] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3375.104084] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3375.132724] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3375.140639] Lustre: Skipped 1 previous similar message [ 3377.999310] Lustre: server umount lustre-MDT0000 complete [ 3381.877383] LustreError: 126419:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398500 with bad export cookie 17187638145356994562 [ 3381.889152] LustreError: 126419:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3381.898280] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3382.274294] Lustre: server umount lustre-MDT0001 complete [ 3391.178450] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 07:35:07 (1787398507) [ 3398.650160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3399.082889] LustreError: 130808:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3399.101684] LustreError: 130808:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3399.134039] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3404.268540] LustreError: 130809:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3405.861388] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3408.353693] LustreError: 130808:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3413.475448] LustreError: 130809:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3416.984742] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3422.742957] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3431.717078] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3438.843168] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3445.958764] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3446.265536] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3446.272454] Lustre: Skipped 1 previous similar message [ 3452.433393] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3459.569729] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3475.174734] Lustre: *** cfs_fail_loc=159, val=0*** [ 3491.426295] Lustre: lustre-MDT0000: Client 281836cc-0821-43d1-b81c-977de8f016b0 (at 192.168.203.36@tcp) reconnecting [ 3492.954278] Lustre: *** cfs_fail_loc=159, val=0*** [ 3492.958043] Lustre: Skipped 5 previous similar messages [ 3508.367674] Lustre: lustre-MDT0000: Client 281836cc-0821-43d1-b81c-977de8f016b0 (at 192.168.203.36@tcp) reconnecting [ 3529.189707] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3529.213270] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3529.230542] Lustre: Skipped 1 previous similar message [ 3529.241304] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3529.256702] Lustre: Skipped 2 previous similar messages [ 3533.648228] Lustre: server umount lustre-OST0000 complete [ 3540.449229] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3540.450977] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3540.511135] LustreError: Skipped 1 previous similar message [ 3540.523114] Lustre: Skipped 1 previous similar message [ 3540.547690] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3540.560977] Lustre: Skipped 2 previous similar messages [ 3544.512554] Lustre: server umount lustre-MDT0000 complete [ 3545.578576] LustreError: 133488:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3545.599391] LustreError: 133488:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3548.768106] LustreError: 131544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398667 with bad export cookie 17187638145356995437 [ 3548.777656] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3548.790846] LustreError: 131544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 3549.280069] Lustre: server umount lustre-MDT0001 complete [ 3562.191992] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 3565.041595] Key type lgssc unregistered [ 3565.427994] LNet: 134609:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3565.448649] LNetError: 134609:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3565.467616] LNet: Removed LNI 192.168.203.136@tcp [ 3566.449633] Key type .llcrypt unregistered [ 3566.452492] Key type ._llcrypt unregistered [ 3592.738387] Key type ._llcrypt registered [ 3592.741182] Key type .llcrypt registered [ 3592.833741] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 3594.135916] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3594.162587] alg: No test for adler32 (adler32-zlib) [ 3595.288811] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 3595.496871] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 3597.184193] Key type lgssc registered [ 3598.469335] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3605.807523] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3605.828448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3607.205792] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3611.132773] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3617.764426] LustreError: 136028:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3617.786226] LustreError: 136028:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3619.246542] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3619.727481] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3624.109249] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3632.483672] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3637.819650] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3644.837279] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3645.269952] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3652.488317] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3652.615876] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3660.803654] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3673.444127] Lustre: *** cfs_fail_loc=159, val=0*** [ 3689.056848] Lustre: lustre-MDT0000: Client 58e40612-592e-4d23-a358-dab357addaec (at 192.168.203.36@tcp) reconnecting [ 3692.025672] Lustre: *** cfs_fail_loc=159, val=0*** [ 3708.509907] Lustre: lustre-MDT0001: Client 58e40612-592e-4d23-a358-dab357addaec (at 192.168.203.36@tcp) reconnecting [ 3710.390748] Lustre: *** cfs_fail_loc=159, val=0*** [ 3710.395094] Lustre: Skipped 3 previous similar messages [ 3726.963196] Lustre: lustre-MDT0001: Client 58e40612-592e-4d23-a358-dab357addaec (at 192.168.203.36@tcp) reconnecting [ 3732.819579] Lustre: 139063:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3736.068382] Lustre: *** cfs_fail_loc=159, val=0*** [ 3736.079133] Lustre: Skipped 4 previous similar messages [ 3752.556730] Lustre: lustre-MDT0000: Client c618eeee-079b-4079-bbe9-3a659febe1a6 (at 192.168.203.36@tcp) reconnecting [ 3754.886436] Lustre: *** cfs_fail_loc=159, val=0*** [ 3754.892587] Lustre: Skipped 14 previous similar messages [ 3770.530067] Lustre: lustre-MDT0000: Client c618eeee-079b-4079-bbe9-3a659febe1a6 (at 192.168.203.36@tcp) reconnecting [ 3772.161721] Lustre: 139350:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3790.818773] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3790.829194] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3790.851603] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3792.874272] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3792.875708] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3792.890393] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3792.911291] Lustre: Skipped 1 previous similar message [ 3794.693867] Lustre: server umount lustre-OST0000 complete [ 3799.014047] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3799.022389] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3799.046063] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3803.107450] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3803.108474] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3804.762767] Lustre: server umount lustre-MDT0000 complete [ 3808.233795] LustreError: 136024:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3808.264878] LustreError: 136024:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3808.382101] LustreError: 139180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398927 with bad export cookie 4591810348449991830 [ 3808.394652] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3808.397033] LustreError: 139180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3808.974783] Lustre: server umount lustre-MDT0001 complete [ 3820.999966] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 3823.820312] Key type lgssc unregistered [ 3824.204702] LNet: 140432:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3824.215757] LNetError: 140432:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3824.243345] LNet: Removed LNI 192.168.203.136@tcp [ 3825.278421] Key type .llcrypt unregistered [ 3825.282856] Key type ._llcrypt unregistered [ 3848.922393] Key type ._llcrypt registered [ 3848.924658] Key type .llcrypt registered [ 3849.029240] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 3849.982750] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3850.046148] alg: No test for adler32 (adler32-zlib) [ 3851.079597] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 3851.341774] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 3852.984601] Key type lgssc registered [ 3853.935651] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3860.820496] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3860.837134] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3862.194781] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3866.504213] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3872.741567] LustreError: 141847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3872.762230] LustreError: 141847:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3874.521552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3874.828891] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3878.943328] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3886.273148] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3891.489229] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3897.472937] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3897.894523] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3899.979966] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3904.464487] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 3908.163772] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3912.409351] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3918.254558] Lustre: 144427:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3921.433516] Lustre: 144476:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3922.993528] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3923.957327] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3927.929228] Lustre: 144623:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3928.704690] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3929.530601] Lustre: 144722:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3933.667135] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3933.672152] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3933.682290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3933.689823] Lustre: Skipped 1 previous similar message [ 3938.789888] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3938.795225] Lustre: Skipped 1 previous similar message [ 3940.092257] Lustre: server umount lustre-OST0000 complete [ 3943.915356] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3943.918491] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3943.928924] Lustre: Skipped 1 previous similar message [ 3943.934036] Lustre: Skipped 1 previous similar message [ 3949.029243] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3949.041789] Lustre: Skipped 2 previous similar messages [ 3954.149871] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3954.163130] Lustre: Skipped 1 previous similar message [ 3958.240158] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3958.526806] Lustre: server umount lustre-MDT0000 complete [ 3959.274228] LustreError: 141848:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3959.295715] LustreError: 141848:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3962.058277] LustreError: 141828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787399080 with bad export cookie 13529653743595333306 [ 3962.061551] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3962.073374] LustreError: 141828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3962.450742] Lustre: server umount lustre-MDT0001 complete [ 3973.585826] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 3976.591947] Key type lgssc unregistered [ 3977.010923] LNet: 145784:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3977.023815] LNetError: 145784:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3977.058901] LNet: Removed LNI 192.168.203.136@tcp [ 3978.037595] Key type .llcrypt unregistered [ 3978.049920] Key type ._llcrypt unregistered [ 4010.241162] Key type ._llcrypt registered [ 4010.244113] Key type .llcrypt registered [ 4010.355354] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 4012.209653] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4012.256141] alg: No test for adler32 (adler32-zlib) [ 4013.334663] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 4013.667559] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 4015.432254] Key type lgssc registered [ 4017.252290] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4027.065355] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4027.102141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4028.988803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4034.511725] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4039.157173] LustreError: 147200:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4039.182672] LustreError: 147200:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4043.373577] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4043.858640] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4048.473909] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4056.972978] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4064.148512] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4071.207628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4071.590369] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4074.683777] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4077.565183] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4085.301702] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4089.031741] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4093.741231] Lustre: *** cfs_fail_loc=159, val=0*** [ 4109.473118] Lustre: lustre-MDT0000: Client 71219041-b0ee-4bd6-b980-1cf871e6aec5 (at 192.168.203.36@tcp) reconnecting [ 4114.916699] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4114.927839] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4114.943183] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4114.944722] Lustre: Skipped 1 previous similar message [ 4114.961619] Lustre: Skipped 1 previous similar message [ 4117.993512] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4118.004834] Lustre: Skipped 1 previous similar message [ 4120.846963] Lustre: server umount lustre-OST0000 complete [ 4125.666858] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4125.673271] LustreError: Skipped 8 previous similar messages [ 4125.678090] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4125.707212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4128.227724] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4128.229104] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4131.729021] Lustre: server umount lustre-MDT0000 complete [ 4133.348538] LustreError: 147200:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4133.379270] LustreError: 147200:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4136.258854] LustreError: 147181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787399255 with bad export cookie 12188465283278586969 [ 4136.273335] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4136.284209] LustreError: 147181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4136.797072] Lustre: server umount lustre-MDT0001 complete [ 4149.556268] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 4152.203895] Key type lgssc unregistered [ 4152.457210] LNet: 150864:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4152.467802] LNetError: 150864:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4152.496585] LNet: Removed LNI 192.168.203.136@tcp [ 4153.630311] Key type .llcrypt unregistered [ 4153.632358] Key type ._llcrypt unregistered [ 4181.693442] Key type ._llcrypt registered [ 4181.699116] Key type .llcrypt registered [ 4181.895050] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 4183.529787] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4183.593584] alg: No test for adler32 (adler32-zlib) [ 4184.830793] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 4185.125422] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 4186.848242] Key type lgssc registered [ 4188.123545] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4196.349132] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4196.395668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4197.869162] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4202.455272] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4208.104474] LustreError: 152281:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4208.120791] LustreError: 152281:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4212.206050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4212.595784] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4217.611923] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4225.993739] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4232.139330] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4239.442323] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4239.997623] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4243.066597] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4247.376884] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4255.172944] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4257.402435] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4280.673133] Lustre: lustre-OST0000: evicting 671f2e56-512f-437c-81da-53dca36a0ff0 (at 192.168.203.36@tcp) by administrative request [ 4280.690491] Lustre: Skipped 1 previous similar message [ 4314.087263] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4314.096466] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4314.103279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4314.117936] Lustre: Skipped 1 previous similar message [ 4314.136859] Lustre: Skipped 1 previous similar message [ 4319.202991] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4319.215605] Lustre: Skipped 1 previous similar message [ 4324.326404] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4324.332536] Lustre: Skipped 1 previous similar message [ 4325.344085] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4325.515863] Lustre: server umount lustre-OST0000 complete [ 4329.448508] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4329.453824] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4329.461496] Lustre: Skipped 1 previous similar message [ 4334.562925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4334.566290] Lustre: Skipped 2 previous similar messages [ 4335.891440] Lustre: server umount lustre-MDT0000 complete [ 4339.692713] LustreError: 153634:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4339.713051] LustreError: 153634:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4339.790220] LustreError: 152261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787399458 with bad export cookie 13714742522256188198 [ 4339.795322] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4339.801088] LustreError: 152261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4340.226972] Lustre: server umount lustre-MDT0001 complete [ 4352.682511] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 4355.919067] Key type lgssc unregistered [ 4356.258757] LNet: 156355:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4356.266355] LNetError: 156355:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4356.288679] LNet: Removed LNI 192.168.203.136@tcp [ 4357.339820] Key type .llcrypt unregistered [ 4357.342124] Key type ._llcrypt unregistered [ 4437.204298] Key type ._llcrypt registered [ 4437.206674] Key type .llcrypt registered [ 4437.357836] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 4438.676459] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4438.746137] alg: No test for adler32 (adler32-zlib) [ 4439.780806] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 4440.073916] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 4441.768274] Key type lgssc registered [ 4443.072543] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4451.339454] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4451.372734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4452.884507] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4458.957428] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4463.078292] LustreError: 158846:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4463.085742] LustreError: 158846:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4468.200498] LustreError: 158847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4469.365063] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4469.894918] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4475.508958] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4483.785067] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4489.740572] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4498.398559] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4498.887529] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4506.361897] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4514.247932] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4514.561433] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4522.424145] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4534.758447] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4534.762746] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4534.777511] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4537.828217] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4537.828841] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4540.580711] Lustre: server umount lustre-OST0000 complete [ 4546.530032] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4546.536939] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4546.548534] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4546.555913] Lustre: Skipped 1 previous similar message [ 4551.997677] Lustre: server umount lustre-MDT0000 complete [ 4553.187232] LustreError: 159577:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4553.227879] LustreError: 159577:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4557.812321] LustreError: 158826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787399676 with bad export cookie 17006013790653547158 [ 4557.818839] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4557.834762] LustreError: 158826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4558.916753] Lustre: server umount lustre-MDT0001 complete [ 4572.721226] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 4577.092636] Key type lgssc unregistered [ 4577.360664] LNet: 162412:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4577.375812] LNetError: 162412:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4578.409242] LNet: Removed LNI 192.168.203.136@tcp [ 4579.406094] Key type .llcrypt unregistered [ 4579.421612] Key type ._llcrypt unregistered [ 4605.935242] Key type ._llcrypt registered [ 4605.939564] Key type .llcrypt registered [ 4606.132414] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 4607.427103] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4607.491458] alg: No test for adler32 (adler32-zlib) [ 4608.590136] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 4608.828381] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 4610.520228] Key type lgssc registered [ 4611.913544] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4627.826818] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 07:55:44 (1787399744) [ 4648.421856] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 4661.648939] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 4704.690607] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4704.722190] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4706.052414] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4706.128460] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4706.277564] Lustre: lustre-MDT0000: new disk, initializing [ 4706.426772] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4706.470061] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4711.719445] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4724.576951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4724.758265] Lustre: 166678:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4724.843783] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4724.849214] Lustre: Skipped 1 previous similar message [ 4724.922833] Lustre: lustre-MDT0001: new disk, initializing [ 4725.070379] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4725.129090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4725.147798] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4731.506451] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4741.960846] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4748.828487] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4756.772021] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4757.174440] Lustre: lustre-OST0000: new disk, initializing [ 4757.180685] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4757.192759] Lustre: 168649:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4757.296201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4763.746492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4763.762722] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4763.872198] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4764.125238] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4774.319158] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4789.732197] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4789.734087] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4789.750287] LustreError: Skipped 1 previous similar message [ 4789.772981] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4789.781047] Lustre: Skipped 1 previous similar message [ 4791.454670] Lustre: server umount lustre-OST0000 complete [ 4796.898654] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4796.907691] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4796.931695] Lustre: Skipped 1 previous similar message [ 4796.943113] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4799.970935] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4799.972219] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4802.653443] Lustre: server umount lustre-MDT0000 complete [ 4806.843769] LustreError: 166669:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787399925 with bad export cookie 3568725948919927823 [ 4806.848890] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4806.856076] LustreError: 166669:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4807.354245] Lustre: server umount lustre-MDT0001 complete [ 4819.866779] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 4822.840494] Key type lgssc unregistered [ 4823.149837] LNet: 170553:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4823.157633] LNetError: 170553:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4823.186235] LNet: Removed LNI 192.168.203.136@tcp [ 4824.224454] Key type .llcrypt unregistered [ 4824.226187] Key type ._llcrypt unregistered [ 4851.294812] Key type ._llcrypt registered [ 4851.297492] Key type .llcrypt registered [ 4851.385081] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 4852.584330] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4852.740968] alg: No test for adler32 (adler32-zlib) [ 4853.822978] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 4854.202455] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 4855.920201] Key type lgssc registered [ 4857.185957] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4866.299465] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4866.323675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4867.851677] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4873.098244] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4877.281557] LustreError: 171971:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4877.309702] LustreError: 171971:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4881.843076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4882.253825] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4887.195731] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4894.801470] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4901.520045] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4908.642346] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4908.963290] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4913.085680] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4915.502652] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 4924.210813] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4935.066595] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4935.077330] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4935.084717] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4935.088745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.106209] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4936.327833] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4936.357153] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4936.372037] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4936.376082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.398389] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4937.641499] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4937.669810] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4937.686193] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4937.691212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.704910] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4938.709363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.794822] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4939.799682] Lustre: Skipped 1 previous similar message [ 4939.820239] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4939.840341] Lustre: Skipped 1 previous similar message [ 4939.849712] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4939.862205] Lustre: Skipped 1 previous similar message [ 4939.869976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.891202] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4939.902581] Lustre: Skipped 1 previous similar message [ 4941.190889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.052316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.152334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.244160] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4944.249975] Lustre: Skipped 3 previous similar messages [ 4944.264146] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4944.272512] Lustre: Skipped 3 previous similar messages [ 4944.276950] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4944.280406] Lustre: Skipped 3 previous similar messages [ 4944.289514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.303401] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4944.312842] Lustre: Skipped 3 previous similar messages [ 4945.132665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.480995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.624781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.715272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.621574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.461530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.562431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.619284] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4952.634512] Lustre: Skipped 7 previous similar messages [ 4952.645808] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4952.664378] Lustre: Skipped 7 previous similar messages [ 4952.672623] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4952.680397] Lustre: Skipped 7 previous similar messages [ 4952.685294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.704515] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4952.716131] Lustre: Skipped 7 previous similar messages [ 4953.537869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.342799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.222460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.272706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.183248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.073556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.943406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.849964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.760468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.948932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.893098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.007287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.278976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.432686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.364033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.425869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.479196] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4969.491752] Lustre: Skipped 16 previous similar messages [ 4969.505589] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4969.514182] Lustre: Skipped 16 previous similar messages [ 4969.518019] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4969.523122] Lustre: Skipped 16 previous similar messages [ 4969.527887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.538428] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4969.557833] Lustre: Skipped 16 previous similar messages [ 4970.366105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.386794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.409419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.265248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.104074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.990792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.883145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.802795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.711390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.571784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.492472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.429614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.381386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.373924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.187229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.136574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.868543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.785287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.684812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.651974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.534897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.403059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.358396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.450095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.302971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.143994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.095650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.041956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.745942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.633346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.498906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.514900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.509364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.326951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.231902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.169389] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5002.174375] Lustre: Skipped 35 previous similar messages [ 5002.192472] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5002.200403] Lustre: Skipped 35 previous similar messages [ 5002.204107] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5002.209729] Lustre: Skipped 35 previous similar messages [ 5002.213627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.222688] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5002.230770] Lustre: Skipped 35 previous similar messages [ 5003.146988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.076966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.110366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.084301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.108368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.944921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.789718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.663508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.690758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.643274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.780767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.867548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.852490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.614255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.548965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.520527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.408235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.292596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.219911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.351274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.276540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.289828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.166871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.195518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.117106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.048301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.989921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.033836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.854414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.742876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.564556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.629522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.460714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.376100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.330574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.315239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.161879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.969061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.773681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.857200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.707878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.550590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.455321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.522500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.478238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.401522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.232461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.027852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.844454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.692410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.835699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.778572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.782562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.699889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.591258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.451376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.330406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.219813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.171861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.905178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.813823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.703222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.569968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.432779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.324574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.228550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.959055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.007663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.800510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.682516] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5066.692500] Lustre: Skipped 69 previous similar messages [ 5066.704322] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5066.716532] Lustre: Skipped 69 previous similar messages [ 5066.719947] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5066.724877] Lustre: Skipped 69 previous similar messages [ 5066.727319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.743418] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5066.749253] Lustre: Skipped 69 previous similar messages [ 5067.620699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.461356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.304049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.199237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.052226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.980437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.861185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.718489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.678317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.629807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.570699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.466879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.652427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.504962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.460951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.339462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.207700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.094166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.164813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.138558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.089520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.033152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.004373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.790453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.795362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.751717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.735773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.648945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.554419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.509373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.535418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.390630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.165883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.980908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.949960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.915977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.832763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.775704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.627327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.437045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.354283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.173686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.130252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.004544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.949130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.677904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.435139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.203614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.366072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.336705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.482593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.447132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.400062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.275519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.303257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.149350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.124069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.067323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.041642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.812570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.789135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.707885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.764575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.965837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.227268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.253845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.439414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.528537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.319269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.384758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.278888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.273925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.081867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.001646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.839640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.871266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.759697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.885559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.911372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.815817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.575465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.576883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.466816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.494616] LustreError: 171991:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5145.501572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.504808] LustreError: 171991:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5146.412435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.370034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.346465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.190929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.267781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.215859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.249642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.139426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.847606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.775294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.580688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.468491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.314493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.266323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.283086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.150788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.047520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.858688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.723752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.579783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.465501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.420310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.264737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.122077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.064773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.843444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.673965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.458081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.303804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.229673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.118742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.935776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.798331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.615836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.358309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.153347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.985743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.975883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.991360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.108570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.096527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.062380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.920978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.816406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.707495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.479295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.456956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.309659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.256565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.238714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.195437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.051486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.053412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.904058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.893581] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5194.907471] Lustre: Skipped 138 previous similar messages [ 5194.935459] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5194.957156] Lustre: Skipped 138 previous similar messages [ 5194.975188] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5194.987110] Lustre: Skipped 138 previous similar messages [ 5194.992372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.000891] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5195.006086] Lustre: Skipped 138 previous similar messages [ 5196.034137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.068992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.022747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.073196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.877718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.796499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.818087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.698270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.744421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.698379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.553084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.547984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.438333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.575533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.350411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.435967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.288531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.396397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.461975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.584502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.625127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.717725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.597903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.875856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.113272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.259910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.585713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.834903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.746203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.981481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.411885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.688260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.789918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.530096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.456855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.594488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.750894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.602803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.608304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.446014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.354089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.232354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.311602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.555548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.651893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.806926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.793255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.645474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.643695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.418656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.274413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.191991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.245568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.986863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.854521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.756596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.856073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.591876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.464922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.362076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.489385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.314874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.242657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.141889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.257855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.258849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.184815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.007414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.950217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.826920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.918257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.817351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.806860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.677868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.446528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.388877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.385353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.359513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.213247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.019860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.853947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.739726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.625877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.471760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.298784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.132465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.928120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.937903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.900963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.848451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.591446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.660406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.685740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.741985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.778531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.712668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.781297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.806558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.705559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.533541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.516197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.575426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.473580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.543481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.291330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.316188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.279131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.229950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.117363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.271390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.261349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.043922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.887859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.888538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.816845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.024650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.015329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.305868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.266455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.157313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.221663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.125827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.955074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.771193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.607838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.615405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.492213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.338974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.163771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.044454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.905351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.826936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.716825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.826140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.496707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.380255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.262203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.178470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.085677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.280468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.326016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.644593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.572802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.340161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.515348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.942816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.866287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.303762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.513920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.804408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.978776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.469767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.141725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.361600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.555168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.602628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.620246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.882315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.898786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.916534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.851836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.896282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.066894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.197786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.001516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.909775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.853650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.924547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.021514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.725642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.759556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.795079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.695303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.565958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.626170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.542499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.421947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.457891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.426049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.319434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.261840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.158681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.179784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.359230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.379235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.403755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.403555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.296754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.155214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.046703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.835669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.757372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.569412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.494476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.548974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.537956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.305971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.296890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.091986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.898271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.701528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.492779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.182821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.950961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.732219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.532049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.410945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.177663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.058616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.104757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.873372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.585778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.370949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.216891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.974751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.818417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.513892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.240285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.054939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.882866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.642154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.466912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.264872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.065389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.843756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.719811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.511800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.370283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.258851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.143618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.906107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.795534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.673469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.461562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.217453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.002455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.762395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.675489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.438241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.306384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.363774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.400028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.252853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.129369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.077965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.852612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.657151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.474326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.295502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.057996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.761069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.536643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.444213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.237436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.159860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.948273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.908993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.761651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.696462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.553730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.421250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.218089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.115165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.922191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.733754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.636081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.658164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.448360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.328194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.186547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.038768] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5451.043665] Lustre: Skipped 270 previous similar messages [ 5451.054488] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5451.062880] Lustre: Skipped 270 previous similar messages [ 5451.066412] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5451.071357] Lustre: Skipped 270 previous similar messages [ 5451.075794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.094639] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5451.102881] Lustre: Skipped 270 previous similar messages [ 5452.020573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.998558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.887192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.686994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.451778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.296591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.115554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.929242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.678673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.532971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.541707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.593425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.510717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.291885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.157910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.058905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.036507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.924324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.804113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.614224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.470929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.298868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.126490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.950071] LustreError: 171991:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5471.961053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.962140] LustreError: 171991:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5472.810800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.809686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.530433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.726359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.779730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.503981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.388718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.298403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.141774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.002788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.880989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.675796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.797894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.961415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.932433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.938574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.000233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.018176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.268760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.422988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.611691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.573323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.576809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.530779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.397566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.297336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.396778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.361607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.516896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.712064] LustreError: 171991:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5501.725335] LustreError: 171991:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5501.737198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.711852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.740646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.709528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.557424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.459603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.253811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.920310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.802311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.589241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.445151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.276360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.182211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.991848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.866537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.705343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.683703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.526930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.468448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.547951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.696975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.697497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.664668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.476613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.293558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.029672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.816364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.581691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.472987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.404371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.247777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.914072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.560655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.281876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.083734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.811698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.638732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.468624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.284837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.195270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.074521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.955708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.707953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.506487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.415683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.216032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.903439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.688484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.536388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.388514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.210697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.127347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.897506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.726907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.635791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.448204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.360781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.210857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.084191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.940473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.856765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.593765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.452853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.333754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.222413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.247920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.299428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.388667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.432942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.306505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.183746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.961794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.804401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.737266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.520305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.325313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.082038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.002735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.869203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.788501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.756160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.705776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.606791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.460117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.313325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.233941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.135064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.135908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.040638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.960586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.845852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.784375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.673341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.565267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.404582] LustreError: 171991:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5583.413683] LustreError: 171991:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5583.424789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.299219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.135875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.902312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.784757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.727834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.650054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.700457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.460483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.223552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.198658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.146523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.020765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.015666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.862356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.742374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.665686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.531849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.282531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.073238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.967860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.721214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.431073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.320404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.130991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.998762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.905977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.788122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.736922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.669622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.485149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.485051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.247713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.128581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.897376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.791296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.565843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.268957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.994452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.780105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.562978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.360468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.228194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.151052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.200364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.268766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.079937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.962951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.798431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.830587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.862179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.845531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.644391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.463814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.202438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.966531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.700746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.567724] LustreError: 171991:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5632.581821] LustreError: 171991:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5632.588920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.366299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.200767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.014211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.916631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.805375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.685565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.387690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.223086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.096831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.864279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.691894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.609586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.327308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.162860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.970758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.797716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.645401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.474769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.393602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.162238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.943157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.725577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.385255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.146328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.917775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.826472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.766105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.012761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.063068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.305703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.227814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.323957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.360026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.317305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.203847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.440843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.441232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.471970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.362312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.483725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.433966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.208375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.257377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.328125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.470246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.497543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.433127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.310102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.164429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.137065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.997100] LustreError: 171991:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5679.011851] LustreError: 171991:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5679.019317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.907990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.759370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.602840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.450529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.255132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.160087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.043419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.054201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.953375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.870466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.736942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.750140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.560630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.454561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.545894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.461740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.327451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.303567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.176196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.196630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.149221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.064958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.913452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.842919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.751309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.727059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.630516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.554176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.380512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.192795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.124450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.034764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.890979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.768885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.656293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.485522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.196886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.957651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.759598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.583133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.486616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.422443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.198414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.265905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.253567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.219428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.141149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.127461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.907566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.217377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.470816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.386731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.239622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.213981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.160045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.142808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.089700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.098515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.506149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.612037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.978570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.059717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.216281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.448931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.666055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.876533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.028543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.110820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.823344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.639289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.468355] LustreError: 171991:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5746.483212] LustreError: 171991:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5746.494223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.619376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.545456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.319552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.192767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.020634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.784593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.840673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.850950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.887059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.962328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.121726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.252425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.342753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.506637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.498917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.519481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.206413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.936318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.896268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.160968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.127280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.180661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.068348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.861322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.642525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.475120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.231476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.064310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.859721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.729442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.578876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.474600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.257083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.178864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.031446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.894296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.694497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.656584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.475889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.309640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.111559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.075339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.822308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.674261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.626399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.370340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.166947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.995288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.834834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.857627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.730173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.584687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.481968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.349379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.202914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.052167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.066493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.964927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.963787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.946333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.840495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.772303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.828831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.914153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.898421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.033879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.999905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.880297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.677702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.477310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.352376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.170047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.169855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.125937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.983278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.789881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.573918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.512644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.493435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.708519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.709287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.739288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.650934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.574620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.408592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.401138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.283278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.364101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.213905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.971812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.758773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.593678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.453776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.522327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.641724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.537540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.435716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.313627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.585157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.397424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.285584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.242472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.126822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.038168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.958903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.902311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.916710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.679431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.725533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.515965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.612364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.373076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.184303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.166243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.209295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.830379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.886127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.812730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.908426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.913839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.872688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.726722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.560225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.842562] LustreError: 185292:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6197.851309] LustreError: 171991:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6857.703776] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6857.725839] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6857.746854] Lustre: Skipped 450 previous similar messages [ 6857.762067] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6859.749143] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6859.770994] Lustre: Skipped 1 previous similar message [ 6864.230058] Lustre: server umount lustre-OST0000 complete [ 6869.198256] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6869.210874] Lustre: Skipped 1 previous similar message [ 6874.594435] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6874.614406] Lustre: Skipped 1 previous similar message [ 6875.768204] Lustre: server umount lustre-MDT0000 complete [ 6879.717451] LustreError: 171967:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6879.757512] LustreError: 171967:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6880.028416] LustreError: 171952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787401998 with bad export cookie 13820489530558100981 [ 6880.040552] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6880.056933] LustreError: 171952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6880.530704] Lustre: server umount lustre-MDT0001 complete [ 6892.138620] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 6894.989060] Key type lgssc unregistered [ 6895.295084] LNet: 227220:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6895.319670] LNetError: 227220:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6895.341830] LNet: Removed LNI 192.168.203.136@tcp [ 6896.259328] Key type .llcrypt unregistered [ 6896.270192] Key type ._llcrypt unregistered [ 6921.638524] Key type ._llcrypt registered [ 6921.642122] Key type .llcrypt registered [ 6921.741387] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 6922.912232] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6923.039944] alg: No test for adler32 (adler32-zlib) [ 6924.218380] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 6924.521201] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 6926.193293] Key type lgssc registered [ 6927.512307] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6935.069837] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6935.114382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6936.697899] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6940.722282] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 6946.274142] LustreError: 228639:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6946.300474] LustreError: 228639:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6949.051481] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6949.449733] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6953.811728] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 6961.798832] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6967.708115] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6974.028707] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6974.325850] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6979.728466] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 6981.514727] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6987.300623] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6992.085294] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6993.797667] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6993.826896] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6993.836409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6993.850967] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6993.852337] LustreError: 228659:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6995.937123] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6995.942225] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6995.968804] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6996.970295] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6996.971888] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6997.002573] Lustre: Skipped 1 previous similar message [ 6999.633341] Lustre: lustre-OST0000: Not available for connect from 192.168.203.36@tcp (stopping) [ 7001.815366] Lustre: server umount lustre-OST0000 complete [ 7013.952985] LustreError: 228659:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7038.178391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7038.453307] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7045.712126] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7046.644190] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7046.683329] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7055.142362] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7057.196553] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7062.061552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7062.075043] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7063.705202] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0bbc2ba3800.ost_server_uuid 50 [ 7065.656744] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0bbc2ba3800.ost_server_uuid in IDLE state after 0 sec [ 7072.231616] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7072.238617] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7072.243745] LustreError: 231846:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7072.283190] Lustre: Skipped 1 previous similar message [ 7072.463734] Lustre: server umount lustre-OST0000 complete [ 7077.345403] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7077.347736] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7077.355306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7077.373663] Lustre: Skipped 1 previous similar message [ 7082.468603] LustreError: 228640:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7082.486220] LustreError: 228640:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7082.593537] Lustre: server umount lustre-MDT0000 complete [ 7086.278754] LustreError: 228619:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787402205 with bad export cookie 17400502485548039085 [ 7086.300076] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7086.303034] LustreError: 228619:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7086.553873] Lustre: server umount lustre-MDT0001 complete [ 7097.256059] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 7100.068462] Key type lgssc unregistered [ 7100.371575] LNet: 233843:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7100.382322] LNetError: 233843:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7100.402784] LNet: Removed LNI 192.168.203.136@tcp [ 7101.402303] Key type .llcrypt unregistered [ 7101.405160] Key type ._llcrypt unregistered [ 7131.422599] Key type ._llcrypt registered [ 7131.425933] Key type .llcrypt registered [ 7131.545569] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 7132.560817] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7132.605052] alg: No test for adler32 (adler32-zlib) [ 7133.662518] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 7133.843729] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 7135.512163] Key type lgssc registered [ 7136.550811] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7146.029952] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7146.066654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7147.647624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7152.100910] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7157.222715] LustreError: 235572:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7157.263555] LustreError: 235572:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7162.346082] LustreError: 235571:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7162.548240] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7163.154941] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7169.194762] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7172.475810] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7179.581154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7179.992230] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7186.531169] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7192.327173] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7195.369864] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7195.596182] Lustre: lustre-OST0001: new disk, initializing [ 7195.601794] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7195.611552] Lustre: 237879:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7195.755115] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7202.563239] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7203.372533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7203.386272] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7203.432078] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7213.275460] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7221.732828] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7239.143745] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7239.164096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7239.166078] Lustre: Skipped 2 previous similar messages [ 7239.182945] Lustre: Skipped 3 previous similar messages [ 7242.504566] Lustre: server umount lustre-MDT0000 complete [ 7244.265764] LustreError: 238668:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7244.296740] LustreError: 238668:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7245.978317] LustreError: 238765:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787402364 with bad export cookie 17002691922590092018 [ 7245.979096] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7245.991956] LustreError: 238765:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7246.469693] Lustre: server umount lustre-MDT0001 complete [ 7260.804930] Lustre: server umount lustre-OST0000 complete [ 7274.830671] Lustre: server umount lustre-OST0001 complete [ 7293.430629] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 7301.628277] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7301.987072] LustreError: 240988:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7302.062417] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7305.254908] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7307.238071] LustreError: 240989:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7307.252089] LustreError: 240989:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7312.973785] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7313.291245] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7317.201559] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7319.775300] Lustre: 242129:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7325.451265] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7326.826407] LustreError: 242486:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7326.834479] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7326.838771] LustreError: 242486:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7331.066336] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7338.123591] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7338.305943] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7338.310703] Lustre: Skipped 1 previous similar message [ 7343.924948] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7350.150443] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7353.737504] Lustre: 244002:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7364.577738] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7364.591274] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7364.614308] Lustre: Skipped 1 previous similar message [ 7364.625346] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7369.639695] Lustre: server umount mylustre-MDT0000 complete [ 7370.216444] LustreError: 240983:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7370.242831] LustreError: 240983:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 7372.961826] LustreError: 244200:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787402491 with bad export cookie 17002691922590094832 [ 7372.965274] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7372.976445] LustreError: 244200:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7373.281131] Lustre: server umount mylustre-MDT0001 complete [ 7386.313031] Lustre: server umount mylustre-OST0000 complete [ 7400.083284] Lustre: server umount mylustre-OST0001 complete [ 7418.125454] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 7426.472254] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7426.903313] LustreError: 246310:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7427.003939] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7431.721578] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7439.535664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7443.929343] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7446.628866] Lustre: 247451:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7452.330836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7457.894048] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7465.755890] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7472.252650] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7473.149734] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7479.623630] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7483.031961] Lustre: 249319:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7491.043425] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7491.053945] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7491.066591] Lustre: Skipped 3 previous similar messages [ 7491.082850] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7491.086820] Lustre: Skipped 4 previous similar messages [ 7493.607142] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7493.615446] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7493.624833] Lustre: Skipped 1 previous similar message [ 7493.636063] Lustre: Skipped 3 previous similar messages [ 7494.868164] Lustre: server umount tfs-MDT0000 complete [ 7498.388954] LustreError: 246287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787402617 with bad export cookie 17002691922590096498 [ 7498.398163] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7498.406598] LustreError: 246287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7498.750409] Lustre: server umount tfs-MDT0001 complete [ 7512.532025] Lustre: server umount tfs-OST0000 complete [ 7515.104194] Lustre: 234837:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787402617/real 1787402617] req@ffff89d050c83b80 x1874227103992064/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787402633 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7515.132400] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7515.142884] Lustre: Skipped 1 previous similar message [ 7516.039579] Lustre: server umount tfs-OST0001 complete [ 7538.305194] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 7548.186588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7548.652807] LustreError: 251567:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7548.677571] LustreError: 251567:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7548.820079] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7548.839832] Lustre: Skipped 3 previous similar messages [ 7553.427697] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7562.157585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7568.619825] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7571.499166] Lustre: 252711:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7579.349473] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7586.739363] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7594.487935] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7596.181251] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7603.838860] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7612.376221] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7615.878261] Lustre: 254582:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7622.702886] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 08:45:39 (1787402739) [ 7639.528386] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7639.556371] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7639.585426] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7644.553781] Lustre: server umount lustre-MDT0000 complete [ 7648.533084] LustreError: 251544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787402767 with bad export cookie 17002691922590097940 [ 7648.544311] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7648.549440] LustreError: 251544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7652.839263] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7652.846742] Lustre: Skipped 5 previous similar messages [ 7655.007806] Lustre: server umount lustre-MDT0001 complete [ 7664.625258] Lustre: server umount lustre-OST0000 complete [ 7674.567583] Lustre: server umount lustre-OST0001 complete [ 7681.156872] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 7690.778132] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 7731.170841] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7731.170841] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7731.170841] [ 7731.224368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7731.469425] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7731.495963] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7731.597983] Lustre: lustre-MDT0000: new disk, initializing [ 7731.712682] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7731.715846] Lustre: Skipped 3 previous similar messages [ 7731.760668] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7736.431634] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7747.397483] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7747.397483] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7747.397483] [ 7747.453218] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7747.614084] Lustre: 257778:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7747.700821] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7747.707128] Lustre: Skipped 1 previous similar message [ 7747.840744] Lustre: lustre-MDT0001: new disk, initializing [ 7748.003119] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7748.014518] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7752.300159] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7762.829633] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7763.027654] Lustre: lustre-OST0000: new disk, initializing [ 7763.030385] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7763.034656] Lustre: 259361:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7764.304839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7764.325338] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7764.511390] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7770.360753] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7780.649977] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7780.752741] Lustre: lustre-OST0001: new disk, initializing [ 7780.755642] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7780.760574] Lustre: 260232:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7782.100979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7782.118042] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7782.278585] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7788.275318] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7799.408891] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 08:48:36 (1787402916) [ 7813.376602] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 08:48:49 (1787402929) [ 7828.458491] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7828.476490] Lustre: Skipped 5 previous similar messages [ 7828.485612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7828.491716] Lustre: Skipped 3 previous similar messages [ 7834.418529] Lustre: server umount lustre-OST0000 complete [ 7838.695885] LustreError: 259347:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7838.717125] LustreError: 259347:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 14 previous similar messages [ 7844.063994] Lustre: server umount lustre-MDT0000 complete [ 7847.696759] LustreError: 260231:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787402966 with bad export cookie 17002691922590099270 [ 7847.711401] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7847.716927] LustreError: 260231:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7862.240193] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7862.550168] Lustre: server umount lustre-MDT0001 complete [ 7872.395553] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 7882.986708] Lustre: server umount lustre-OST0001 complete [ 7890.003523] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 7904.557819] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 7940.113805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7940.452064] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7940.477427] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7940.548459] Lustre: lustre-MDT0000: new disk, initializing [ 7940.650250] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7944.949355] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7954.844696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7954.956290] Lustre: 264798:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7955.288353] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7959.575870] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7968.860429] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7973.762165] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7980.049702] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7980.323287] Lustre: lustre-OST0000: new disk, initializing [ 7980.326297] Lustre: Skipped 1 previous similar message [ 7980.331220] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7980.335533] Lustre: Skipped 2 previous similar messages [ 7980.339871] Lustre: 266765:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7982.249913] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7982.268976] Lustre: Skipped 1 previous similar message [ 7982.278605] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7982.417391] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7986.167881] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 7996.153548] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8003.042910] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8003.050601] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8003.059098] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8003.073891] Lustre: Skipped 6 previous similar messages [ 8003.101481] Lustre: Skipped 12 previous similar messages [ 8008.655577] Lustre: server umount lustre-OST0000 complete [ 8018.257375] Lustre: server umount lustre-MDT0000 complete [ 8022.265716] LustreError: 265593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787403141 with bad export cookie 17002691922590100740 [ 8022.276918] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8022.295521] LustreError: 265593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8022.738162] Lustre: server umount lustre-MDT0001 complete [ 8032.536480] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 8035.355203] Key type lgssc unregistered [ 8035.697440] LNet: 268528:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8035.711317] LNetError: 268528:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8035.752471] LNet: Removed LNI 192.168.203.136@tcp [ 8036.576165] Key type .llcrypt unregistered [ 8036.581032] Key type ._llcrypt unregistered [ 8069.027816] Key type ._llcrypt registered [ 8069.030166] Key type .llcrypt registered [ 8069.208475] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 8071.096143] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8071.288177] alg: No test for adler32 (adler32-zlib) [ 8072.405404] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 8072.652613] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 8074.312316] Key type lgssc registered [ 8075.778372] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8084.826867] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8084.871932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8086.231874] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8091.158484] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8096.743152] LustreError: 269991:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8096.766108] LustreError: 269991:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8101.023674] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8101.556600] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8106.810389] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8114.953701] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8121.448671] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8129.595984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8130.065380] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8132.180900] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8137.367443] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8145.784925] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8146.873421] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8151.527394] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8151.530542] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8151.556254] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8155.393969] Lustre: server umount lustre-OST0000 complete [ 8161.779288] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8161.795357] Lustre: Skipped 1 previous similar message [ 8161.805215] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8161.815161] Lustre: Skipped 2 previous similar messages [ 8165.870588] Lustre: server umount lustre-MDT0000 complete [ 8166.896169] LustreError: 269992:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8166.914626] LustreError: 269992:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8169.441551] LustreError: 272521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787403288 with bad export cookie 10202703975521365209 [ 8169.444214] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8169.458257] LustreError: 272521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8169.932180] Lustre: server umount lustre-MDT0001 complete [ 8182.115671] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 8184.821942] Key type lgssc unregistered [ 8185.053675] LNet: 273577:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8185.061181] LNetError: 273577:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8186.086833] LNet: Removed LNI 192.168.203.136@tcp [ 8187.445860] Key type .llcrypt unregistered [ 8187.447793] Key type ._llcrypt unregistered [ 8206.721969] Key type ._llcrypt registered [ 8206.727453] Key type .llcrypt registered [ 8206.868543] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 8208.010646] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8208.033936] alg: No test for adler32 (adler32-zlib) [ 8209.288962] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 8209.634800] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 8211.424162] Key type lgssc registered [ 8212.885084] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8223.384695] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8223.404712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8238.944444] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8244.064685] LustreError: 274774:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.136@tcp: failed processing log, type 4: rc = -110 [ 8270.880205] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8278.093891] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8286.944344] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8289.782495] Lustre: server umount lustre-OST0000 complete [ 8306.364249] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 8309.256931] Key type lgssc unregistered [ 8309.560744] LNet: 276405:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8309.568517] LNetError: 276405:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8309.592666] LNet: Removed LNI 192.168.203.136@tcp [ 8310.455617] Key type .llcrypt unregistered [ 8310.458492] Key type ._llcrypt unregistered [ 8351.884317] Key type ._llcrypt registered [ 8351.886934] Key type .llcrypt registered [ 8352.040182] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 8354.074685] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8354.217712] alg: No test for adler32 (adler32-zlib) [ 8355.375674] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 8355.650513] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 8357.297631] Key type lgssc registered [ 8359.387185] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8373.837377] loop0: detected capacity change from 0 to 204800000 [ 8374.074902] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8374.093117] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8374.378365] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8375.361622] loop0: detected capacity change from 0 to 204800000 [ 8375.573972] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8375.593932] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8376.024511] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8377.203915] loop0: detected capacity change from 0 to 204800000 [ 8377.370480] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8377.380569] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8377.544780] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8379.025627] loop0: detected capacity change from 0 to 204800000 [ 8379.344928] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8379.362858] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8379.477934] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8380.655484] loop0: detected capacity change from 0 to 204800000 [ 8380.705521] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8385.418974] loop0: detected capacity change from 0 to 204800000 [ 8385.472118] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8390.023978] loop0: detected capacity change from 0 to 204800000 [ 8390.042657] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8394.002763] loop0: detected capacity change from 0 to 204800000 [ 8394.029536] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8397.987599] loop0: detected capacity change from 0 to 204800000 [ 8398.087630] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8398.148929] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8398.219322] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8398.226462] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8398.235825] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8398.244429] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8398.252139] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8398.259925] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8398.268866] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8398.282645] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8398.302204] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8398.333314] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8398.343085] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8398.351273] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8398.358134] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8398.368700] Lustre: 279820:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8403.713578] Lustre: server umount MGS complete [ 8404.519191] loop0: detected capacity change from 0 to 204800000 [ 8404.623416] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8405.098603] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8405.155879] Lustre: lustre-MDT0000: Aborting client recovery [ 8406.061848] loop1: detected capacity change from 0 to 204800000 [ 8406.240303] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8406.346906] Lustre: 280253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8406.352723] Lustre: 280253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8406.361604] Lustre: 280253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8406.368957] Lustre: 280253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8406.378229] Lustre: 280253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8406.385564] Lustre: 280253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8406.392693] Lustre: 280253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8406.407212] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8406.554043] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8406.604299] Lustre: lustre-MDT0001: Aborting client recovery [ 8407.585903] loop2: detected capacity change from 0 to 204800000 [ 8407.743161] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8407.798438] Lustre: 280332:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8407.806862] Lustre: 280332:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8407.815708] Lustre: 280332:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8407.837237] Lustre: 280332:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8407.848781] Lustre: 280332:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8407.856840] Lustre: 280332:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8407.875584] Lustre: 280332:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8408.013826] Lustre: Found index 0 for lustre-OST0000, updating log [ 8408.049845] Lustre: 280351:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8408.156749] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8408.195426] Lustre: lustre-OST0000: Aborting recovery [ 8409.047271] loop3: detected capacity change from 0 to 204800000 [ 8409.260101] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8409.378758] Lustre: 280417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8409.394342] Lustre: 280417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8409.415779] Lustre: 280417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8409.426355] Lustre: 280417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8409.433908] Lustre: 280417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8409.442674] Lustre: 280417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8409.450917] Lustre: 280417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8409.469845] Lustre: Found index 1 for lustre-OST0001, updating log [ 8409.494787] Lustre: 280424:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8409.557779] Lustre: lustre-OST0001: Aborting recovery [ 8414.713545] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8414.740582] LustreError: 280196:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8414.756572] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8414.811174] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8414.857743] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8416.813778] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8416.818810] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8417.613046] LustreError: 280163:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3237us: rc = -2 [ 8418.141705] LustreError: 280162:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1467us: rc = -2 [ 8418.161621] LustreError: 280162:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 23 previous similar messages [ 8432.099526] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8432.105154] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8432.105967] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8432.121041] Lustre: Skipped 3 previous similar messages [ 8432.138965] Lustre: Skipped 3 previous similar messages [ 8437.199191] Lustre: server umount lustre-MDT0000 complete [ 8437.233346] LustreError: 280166:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8437.276118] LustreError: 280166:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 8438.167965] LustreError: 280146:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787403557 with bad export cookie 9591175867652640422 [ 8438.192974] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8442.341573] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8442.343387] LustreError: 280162:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8442.350183] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8442.370380] Lustre: Skipped 1 previous similar message [ 8442.396612] LustreError: 280162:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8443.361413] LustreError: 280162:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8443.361708] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8443.384418] LustreError: 280162:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8443.404510] Lustre: Skipped 2 previous similar messages [ 8444.664661] Lustre: server umount lustre-MDT0001 complete [ 8460.256131] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8460.400915] Lustre: server umount lustre-OST0000 complete [ 8467.406200] Lustre: server umount lustre-OST0001 complete [ 8476.773489] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 08:59:53 (1787403593) [ 8508.460336] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 8518.692793] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 8556.595802] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8556.862349] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8556.888833] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8556.965319] Lustre: lustre-MDT0000: new disk, initializing [ 8557.059233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8557.064707] Lustre: Skipped 1 previous similar message [ 8557.082981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8561.946652] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8572.270677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8572.463975] Lustre: 284503:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8572.556861] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8572.561491] Lustre: Skipped 1 previous similar message [ 8572.681437] Lustre: lustre-MDT0001: new disk, initializing [ 8572.825887] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8572.877392] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8572.891041] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8577.187569] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8586.931326] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8592.703713] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8600.402315] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8600.790721] Lustre: lustre-OST0000: new disk, initializing [ 8600.795238] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8600.807771] Lustre: 286469:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8600.976847] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8607.805443] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8609.403357] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8609.422160] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8609.562117] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8621.096447] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8628.369654] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8637.671725] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8643.680120] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8649.836936] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8691.684600] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8691.684705] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8691.685856] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8691.690015] LustreError: Skipped 1 previous similar message [ 8691.704022] Lustre: Skipped 1 previous similar message [ 8694.941245] Lustre: server umount lustre-OST0000 complete [ 8700.905481] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8700.913457] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8700.929886] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8700.937910] Lustre: Skipped 1 previous similar message [ 8705.460114] Lustre: server umount lustre-MDT0000 complete [ 8707.042061] LustreError: 285733:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8707.055142] LustreError: 285733:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8709.013268] LustreError: 287825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787403827 with bad export cookie 9591175867652643131 [ 8709.024371] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8709.025786] LustreError: 287825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 8709.355221] Lustre: server umount lustre-MDT0001 complete [ 8717.527369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8721.870699] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8726.536760] Lustre: server umount MGS complete [ 8737.549866] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8738.137278] LustreError: 290246:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8738.166236] LustreError: 290259:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8738.223525] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8742.753526] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8748.515504] LustreError: 290247:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8748.564089] LustreError: 290247:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8752.443901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8757.396588] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8764.709330] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8771.361192] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8778.113973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8778.436129] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8778.445347] Lustre: Skipped 1 previous similar message [ 8785.232138] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8792.438555] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8823.781598] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8823.786678] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8823.787521] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8823.787527] Lustre: Skipped 2 previous similar messages [ 8823.793587] Lustre: Skipped 2 previous similar messages [ 8829.898630] Lustre: server umount lustre-OST0000 complete [ 8834.023894] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8834.045204] Lustre: Skipped 1 previous similar message [ 8839.883570] Lustre: server umount lustre-MDT0000 complete [ 8843.416378] LustreError: 290226:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787403962 with bad export cookie 9591175867652645056 [ 8843.422912] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8843.431606] LustreError: 290226:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8843.786188] Lustre: server umount lustre-MDT0001 complete [ 8855.074841] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 8857.630976] Key type lgssc unregistered [ 8857.980204] LNet: 294194:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8857.990942] LNetError: 294194:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8858.017361] LNet: Removed LNI 192.168.203.136@tcp [ 8858.994398] Key type .llcrypt unregistered [ 8858.997332] Key type ._llcrypt unregistered [ 8899.032372] Key type ._llcrypt registered [ 8899.038066] Key type .llcrypt registered [ 8899.255416] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 8917.589449] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 8918.827215] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8918.916203] alg: No test for adler32 (adler32-zlib) [ 8920.095192] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 8920.382179] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 8922.137339] Key type lgssc registered [ 8923.575865] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8962.517875] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8962.552520] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8963.899392] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8963.942416] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8964.087810] Lustre: lustre-MDT0000: new disk, initializing [ 8964.194656] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8964.221205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8968.492117] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8979.903930] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8980.076921] Lustre: 298152:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8980.134875] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8980.140708] Lustre: Skipped 1 previous similar message [ 8980.256145] Lustre: lustre-MDT0001: new disk, initializing [ 8980.452659] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8980.508247] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8980.520474] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8985.358606] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 8996.626567] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9003.325805] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9010.252848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9010.599441] Lustre: lustre-OST0000: new disk, initializing [ 9010.606841] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9010.617669] Lustre: 300123:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9010.709930] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9015.927555] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9015.944441] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9016.029252] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9018.127194] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 9027.121502] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9034.789621] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9046.397834] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9054.076252] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9103.329672] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9103.336100] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9103.354211] Lustre: Skipped 1 previous similar message [ 9103.359182] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9103.361557] Lustre: Skipped 1 previous similar message [ 9108.451360] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9108.462514] Lustre: Skipped 1 previous similar message [ 9109.411983] Lustre: server umount lustre-OST0000 complete [ 9113.569886] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9113.571939] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9113.589414] Lustre: Skipped 1 previous similar message [ 9118.696262] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9118.702361] Lustre: Skipped 2 previous similar messages [ 9123.812412] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9123.820945] Lustre: Skipped 1 previous similar message [ 9127.392428] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9127.654066] Lustre: server umount lustre-MDT0000 complete [ 9131.490165] LustreError: 301740:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787404250 with bad export cookie 15978645948679048975 [ 9131.493455] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9131.501515] LustreError: 301740:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9131.884711] Lustre: server umount lustre-MDT0001 complete [ 9140.383112] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9144.855890] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 9149.883708] Lustre: server umount MGS complete [ 9162.474051] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9163.131764] LustreError: 304052:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9163.146329] LustreError: 304065:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9163.148323] LustreError: 304052:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9163.228137] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9168.353760] LustreError: 304053:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9169.161830] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 9173.477690] LustreError: 304052:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9178.595947] LustreError: 304053:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9179.185169] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9179.801674] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9184.512849] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 9191.546949] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9197.109285] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9203.883645] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9204.276878] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9210.135368] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 9216.855683] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9244.130444] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9244.132087] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9244.145711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9249.848469] Lustre: server umount lustre-OST0000 complete [ 9254.371409] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9254.398552] Lustre: Skipped 1 previous similar message [ 9258.694216] Lustre: server umount lustre-MDT0000 complete [ 9259.494447] LustreError: 304049:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9259.513601] LustreError: 304049:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9262.311869] LustreError: 304032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787404381 with bad export cookie 15978645948679050816 [ 9262.320436] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9262.324884] LustreError: 304032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9262.696055] Lustre: server umount lustre-MDT0001 complete [ 9272.192422] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [ 9274.661197] Key type lgssc unregistered [ 9274.898875] LNet: 307993:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9274.912694] LNetError: 307993:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9275.949939] LNet: Removed LNI 192.168.203.136@tcp [ 9276.704230] Key type .llcrypt unregistered [ 9276.706657] Key type ._llcrypt unregistered [ 9294.981749] Key type ._llcrypt registered [ 9294.987704] Key type .llcrypt registered [ 9295.101191] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [ 9306.844149] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 9308.097221] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9308.127290] alg: No test for adler32 (adler32-zlib) [ 9309.231594] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [ 9309.456312] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [ 9311.104168] Key type lgssc registered [ 9312.087164] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9334.455150] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [ 9347.695429] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9347.721827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9348.942143] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9348.978941] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9349.091961] Lustre: lustre-MDT0000: new disk, initializing [ 9349.177919] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9349.191545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9353.941338] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [ 9364.207930] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9364.572624] Lustre: lustre-OST0000: new disk, initializing [ 9364.582796] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9364.587592] Lustre: Skipped 1 previous similar message [ 9364.592646] Lustre: 312019:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9364.707585] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9368.296319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9368.304468] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9368.350918] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9370.617615] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10213.346724] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10213.362879] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10216.410729] Lustre: server umount lustre-MDT0000 complete [10220.195352] LustreError: 311207:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787405339 with bad export cookie 7599096584768842570 [10220.208659] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10220.210716] LustreError: 311207:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10220.411434] Lustre: server umount lustre-OST0000 complete [10248.537879] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [10260.038152] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [10300.823855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10301.020560] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10301.057388] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10301.119801] Lustre: lustre-MDT0000: new disk, initializing [10301.213635] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10301.264552] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10306.461981] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10317.997194] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10318.123849] Lustre: 316355:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10318.186942] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10318.189695] Lustre: Skipped 1 previous similar message [10318.319896] Lustre: lustre-MDT0001: new disk, initializing [10318.481505] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10318.554669] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10318.571174] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10323.745809] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10333.267484] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10338.883850] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10346.244199] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10346.596176] Lustre: lustre-OST0000: new disk, initializing [10346.607328] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10346.625341] Lustre: 318323:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10346.770266] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10353.104220] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10354.791904] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10354.808243] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10354.926921] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10362.973741] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10368.376858] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10368.597645] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10373.512460] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10373.789480] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10380.791753] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10380.792871] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10380.795032] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10380.822453] Lustre: Skipped 1 previous similar message [10385.904688] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10385.911542] Lustre: Skipped 2 previous similar messages [10389.984661] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10390.177840] Lustre: server umount lustre-OST0000 complete [10395.621447] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10395.632336] LustreError: Skipped 1 previous similar message [10395.636911] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10395.664960] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10400.662389] Lustre: server umount lustre-MDT0000 complete [10404.162983] LustreError: 316348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787405523 with bad export cookie 7599096584771783396 [10404.163807] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10404.187265] LustreError: 316348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10404.568779] Lustre: server umount lustre-MDT0001 complete [10408.451495] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10410.568419] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 09:32:06 (1787405526) [10418.507176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10419.032886] LustreError: 320636:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10419.051837] LustreError: 320636:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10419.157566] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10424.065872] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10424.294818] LustreError: 320637:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10428.387746] LustreError: 320636:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10433.507805] LustreError: 320637:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10433.773437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10434.273628] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10438.537031] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10445.737963] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10452.102748] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10459.758849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10460.162870] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10466.724864] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10473.962209] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10480.859624] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10481.177352] Lustre: lustre-OST0001: new disk, initializing [10481.182270] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10481.189515] Lustre: 323473:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10481.291471] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10487.622114] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10488.429336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10488.436980] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10488.500743] LustreError: 323856:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10489.575187] LustreError: 323856:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10489.599485] LustreError: 323856:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10490.631149] LustreError: 323863:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10492.721226] LustreError: 323856:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10492.736171] LustreError: 323856:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [10496.806640] LustreError: 323863:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10496.819021] LustreError: 323863:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10498.031680] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10498.295986] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10503.881393] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0bbc7e20800.ost_server_uuid 50 [10505.076474] LustreError: 323856:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10505.084177] LustreError: 323856:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10505.378954] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0bbc7e20800.ost_server_uuid in FULL state after 0 sec [10510.994712] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0bbc7e20800.ost_server_uuid 50 [10512.496641] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa0bbc7e20800.ost_server_uuid in FULL state after 0 sec [10514.423781] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10528.843463] Lustre: lustre-OST0001: Not available for connect from 192.168.203.36@tcp (stopping) [10528.849304] Lustre: Skipped 2 previous similar messages [10529.765499] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10529.776183] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10529.796588] Lustre: Skipped 2 previous similar messages [10534.506979] Lustre: server umount lustre-OST0001 complete [10534.888662] LustreError: 322460:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10534.922934] LustreError: 322460:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10545.121474] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10545.128909] LustreError: Skipped 1 previous similar message [10545.130328] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10545.132859] LustreError: 322460:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10545.132869] LustreError: 322460:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10545.142768] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10545.164724] Lustre: Skipped 1 previous similar message [10545.205236] Lustre: Skipped 4 previous similar messages [10547.324763] Lustre: server umount lustre-OST0000 complete [10552.289192] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10555.380422] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10555.402598] Lustre: Skipped 1 previous similar message [10557.279907] Lustre: server umount lustre-MDT0000 complete [10560.795680] LustreError: 323472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787405679 with bad export cookie 7599096584771784278 [10560.802263] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10560.806801] LustreError: 323472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10561.142935] Lustre: server umount lustre-MDT0001 complete [10570.914420] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [10573.424733] Key type lgssc unregistered [10573.740774] LNet: 326162:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10573.763727] LNetError: 326162:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10573.782159] LNet: Removed LNI 192.168.203.136@tcp [10574.707168] Key type .llcrypt unregistered [10574.709982] Key type ._llcrypt unregistered [10603.626681] Key type ._llcrypt registered [10603.631564] Key type .llcrypt registered [10603.764762] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [10604.817825] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10604.856688] alg: No test for adler32 (adler32-zlib) [10605.969201] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [10606.154365] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [10607.808323] Key type lgssc registered [10609.669231] Lustre: Echo OBD driver; http://www.lustre.org/ [10620.031956] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10620.074310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10621.690927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10627.275493] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10631.147115] LustreError: 327587:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10631.179412] LustreError: 327587:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10635.767653] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10636.265350] LustreError: 327586:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10641.379792] LustreError: 327587:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10642.219581] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10642.633470] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10647.113510] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10657.813832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10658.211377] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10664.936239] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10666.419241] LustreError: 329268:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10666.460061] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10672.620702] LustreError: 329707:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10672.645805] LustreError: 329707:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10673.233752] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10681.380252] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10681.571227] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10687.003197] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10688.384609] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10695.507319] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10702.014878] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10702.286694] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10809.826744] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10809.833892] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10809.843931] LustreError: Skipped 1 previous similar message [10809.880582] Lustre: Skipped 1 previous similar message [10809.896036] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10813.834945] Lustre: server umount lustre-OST0001 complete [10814.948667] LustreError: 329896:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10814.970022] LustreError: 329896:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10820.071942] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10820.076556] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10820.077520] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10820.077527] Lustre: Skipped 1 previous similar message [10820.122619] Lustre: Skipped 1 previous similar message [10825.199860] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10825.207937] Lustre: Skipped 2 previous similar messages [10830.317615] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10830.332984] Lustre: Skipped 1 previous similar message [10833.376353] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10833.628445] Lustre: server umount lustre-OST0000 complete [10840.551259] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10840.553321] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10840.578651] Lustre: Skipped 1 previous similar message [10840.598099] Lustre: Skipped 1 previous similar message [10844.384782] Lustre: server umount lustre-MDT0000 complete [10845.665040] LustreError: 331273:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10845.678912] LustreError: 331273:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [10848.636502] LustreError: 328519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787405967 with bad export cookie 8603161827664392196 [10848.643663] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10848.666340] LustreError: 328519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10849.285926] Lustre: server umount lustre-MDT0001 complete [10863.123635] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [10866.400103] Key type lgssc unregistered [10866.775889] LNet: 332533:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10866.790466] LNetError: 332533:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10866.835238] LNet: Removed LNI 192.168.203.136@tcp [10867.988397] Key type .llcrypt unregistered [10867.991548] Key type ._llcrypt unregistered [10910.984436] Key type ._llcrypt registered [10910.987517] Key type .llcrypt registered [10911.099750] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [10911.962331] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10912.092602] alg: No test for adler32 (adler32-zlib) [10913.149240] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [10913.379825] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [10915.048172] Key type lgssc registered [10916.094706] Lustre: Echo OBD driver; http://www.lustre.org/ [10924.896869] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10924.909325] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10926.403622] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10930.390446] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10936.806805] LustreError: 334825:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10936.828894] LustreError: 334825:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10939.390333] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10939.789213] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10944.134707] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10946.680633] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10952.770488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10953.041933] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10958.910093] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10964.264100] LustreError: 336315:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10966.940765] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10967.265556] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10971.406881] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10971.432269] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10973.628959] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [10982.260729] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10990.226401] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11001.315930] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11001.321094] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11001.327038] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11002.340589] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11002.345547] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11002.353203] Lustre: Skipped 2 previous similar messages [11002.360679] Lustre: Skipped 3 previous similar messages [11006.740107] Lustre: server umount lustre-MDT0000 complete [11007.457765] LustreError: 334825:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11007.486933] LustreError: 334825:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11010.592146] LustreError: 335963:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787406129 with bad export cookie 16194626026335490194 [11010.603873] LustreError: 335963:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11010.604798] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11010.876522] Lustre: server umount lustre-MDT0001 complete [11025.298348] Lustre: server umount lustre-OST0000 complete [11028.450482] Lustre: 334090:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787406131/real 1787406131] req@ffff89d04a7d3b80 x1874231067472384/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787406147 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11028.501675] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11029.804490] Lustre: server umount lustre-OST0001 complete [11050.350724] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [11060.165064] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11060.434359] LustreError: 340088:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11060.509281] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11064.715288] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11065.830251] LustreError: 340089:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11072.941405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11073.438722] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11077.710780] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11080.490711] Lustre: 341229:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11086.947296] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11092.471275] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11098.615343] LustreError: 342277:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11098.642838] LustreError: 342277:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11100.861820] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11101.328559] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11101.336378] Lustre: Skipped 1 previous similar message [11105.426598] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11105.433344] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11109.141548] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11116.651171] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11120.111620] Lustre: 343102:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11124.196505] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11124.200197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11124.217827] Lustre: Skipped 1 previous similar message [11129.321636] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11129.327388] Lustre: Skipped 5 previous similar messages [11130.219524] Lustre: server umount lustre-MDT0000 complete [11133.459256] LustreError: 340069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787406252 with bad export cookie 16194626026335491895 [11133.463106] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11133.470154] LustreError: 340069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11133.768770] Lustre: server umount lustre-MDT0001 complete [11147.729342] Lustre: server umount lustre-OST0000 complete [11150.816467] Lustre: 334087:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787406253/real 1787406253] req@ffff89d052a45180 x1874231067520640/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787406269 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11150.845968] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11150.856401] Lustre: Skipped 2 previous similar messages [11151.367517] Lustre: server umount lustre-OST0001 complete [11154.629293] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11156.117029] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11158.010749] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 09:44:34 (1787406274) [11178.487939] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [11187.516944] loop0: detected capacity change from 0 to 1099511627776 [11188.138630] SGI XFS with ACLs, security attributes, quota, no debug enabled [11188.153604] *** VALIDATE xfs *** [11188.175406] XFS (loop0): Mounting V5 Filesystem [11188.206809] XFS (loop0): Ending clean mount [11189.242697] loop1: detected capacity change from 0 to 18691697672192 [11297.395658] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [11307.405596] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [11346.880055] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11347.251332] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11347.306812] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11347.427180] Lustre: lustre-MDT0000: new disk, initializing [11347.584574] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11347.656280] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11353.760228] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11364.495437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11364.649996] Lustre: 348236:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11364.698086] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11364.702318] Lustre: Skipped 1 previous similar message [11364.752604] Lustre: lustre-MDT0001: new disk, initializing [11364.865365] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11364.878274] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11370.044358] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11379.769442] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11385.529399] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11393.026486] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11393.523836] Lustre: lustre-OST0000: new disk, initializing [11393.530178] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11393.545447] Lustre: 350201:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11395.005545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11395.026489] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11395.189096] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11401.785213] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11413.042904] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11418.714202] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11418.966318] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11423.895869] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11424.196047] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11430.892243] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11430.897764] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11430.904109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11430.921563] Lustre: Skipped 1 previous similar message [11430.942048] LustreError: Skipped 1 previous similar message [11432.197495] Lustre: server umount lustre-OST0000 complete [11436.515190] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11439.011191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11439.014508] Lustre: Skipped 3 previous similar messages [11442.420116] Lustre: server umount lustre-MDT0000 complete [11445.223178] LustreError: 348247:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11445.252557] LustreError: 348247:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11446.037298] LustreError: 348227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787406564 with bad export cookie 16194626026335493449 [11446.038954] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11446.059206] LustreError: 348227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11446.551395] Lustre: server umount lustre-MDT0001 complete [11455.367462] XFS (loop0): Unmounting Filesystem [11459.446068] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 09:49:36 (1787406576) [11465.028350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11465.450466] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11465.457781] Lustre: Skipped 2 previous similar messages [11469.237419] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11477.704570] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11482.750936] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11490.135140] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11495.531535] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11501.609565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11508.556679] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11517.225384] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11528.674330] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11528.682488] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11528.700149] Lustre: Skipped 2 previous similar messages [11528.707591] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11528.716029] Lustre: Skipped 2 previous similar messages [11532.377836] Lustre: server umount lustre-OST0000 complete [11539.937628] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11539.944745] LustreError: Skipped 1 previous similar message [11542.469639] Lustre: server umount lustre-MDT0000 complete [11546.028240] LustreError: 352741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787406664 with bad export cookie 16194626026335494331 [11546.043852] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11546.447954] Lustre: server umount lustre-MDT0001 complete [11557.424852] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [11560.330431] Key type lgssc unregistered [11560.619297] LNet: 356392:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11560.626944] LNetError: 356392:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11560.641555] LNet: Removed LNI 192.168.203.136@tcp [11561.422596] Key type .llcrypt unregistered [11561.425067] Key type ._llcrypt unregistered [11589.930877] Key type ._llcrypt registered [11589.933371] Key type .llcrypt registered [11590.062906] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [11591.190718] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11591.251180] alg: No test for adler32 (adler32-zlib) [11592.359394] Lustre: Lustre: Build Version: 2.17.54_108_g575fb00 [11592.625487] LNet: Added LNI 192.168.203.136@tcp [8/256/0/180] [11594.290084] Key type lgssc registered [11595.739747] Lustre: Echo OBD driver; http://www.lustre.org/ [11604.539769] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11604.573065] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11606.045786] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11610.408390] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11615.208929] LustreError: 357962:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11615.221751] LustreError: 357962:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11619.657771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11620.104474] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11625.059885] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11632.707752] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11638.694569] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11645.838408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11646.357331] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11653.259598] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11661.690677] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11674.699716] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.36@tcp (stopping) [11675.113911] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11678.305954] Lustre: server umount lustre-MDT0001 complete [11679.780370] LustreError: 357958:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11684.907313] LustreError: 357956:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11684.940109] LustreError: 357956:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11688.872901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11689.099736] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11689.102152] Lustre: Found index 1 for lustre-MDT0001, updating log [11689.221587] Lustre: 357950:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11689.757062] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11695.096146] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11695.558488] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11696.114644] LustreError: 361430:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11696.118776] LustreError: 361430:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11696.124082] LustreError: 357954:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11696.211682] LustreError: 361433:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.136@tcp: cfg command failed: rc = -17 [11696.218414] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11696.218414] [11710.435498] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11710.444751] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11710.473762] Lustre: Skipped 1 previous similar message [11710.486979] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11710.489281] Lustre: Skipped 2 previous similar messages [11712.938776] Lustre: server umount lustre-MDT0001 complete [11715.554962] LustreError: 357962:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11715.593747] LustreError: 357962:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11720.681298] LustreError: 358690:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11720.700444] LustreError: 358690:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11724.212218] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11724.360144] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11724.365578] Lustre: Found index 1 for lustre-MDT0001, updating log [11724.467728] Lustre: 357950:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11724.871396] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11728.990368] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [11729.908424] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11730.954529] LustreError: 362589:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11730.966368] LustreError: 362589:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11730.974813] LustreError: 357954:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11730.989465] LustreError: 357954:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11731.082215] LustreError: 362600:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11731.093423] LustreError: 362600:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11731.108819] LustreError: 362600:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.136@tcp: cfg command failed: rc = -17 [11731.116482] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.136@tcp [11731.116482] [12037.091766] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12037.096366] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12037.112450] Lustre: Skipped 1 previous similar message [12037.113645] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12037.113650] Lustre: Skipped 1 previous similar message [12040.730173] Lustre: server umount lustre-MDT0001 complete [12042.229705] LustreError: 357962:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12042.244226] LustreError: 357962:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12052.454289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12052.513601] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12052.518482] Lustre: Found index 1 for lustre-MDT0001, updating log [12052.542828] Lustre: 357951:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12052.771341] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12057.953942] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [12058.103194] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12058.609218] LustreError: 371280:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12058.626753] LustreError: 371280:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12058.642133] LustreError: 357954:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12058.649288] LustreError: 357954:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12058.733792] LustreError: 371286:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12058.772387] LustreError: 371286:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.136@tcp: cfg command failed: rc = -114 [12058.795371] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12058.795371] [12380.642105] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12380.646737] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12380.662743] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12380.681695] Lustre: Skipped 2 previous similar messages [12380.707792] Lustre: Skipped 4 previous similar messages [12385.474630] Lustre: server umount lustre-MDT0000 complete [12385.761373] LustreError: 357957:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12385.788742] LustreError: 357957:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [12389.049727] LustreError: 357943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787407507 with bad export cookie 5005524566908657280 [12389.053393] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12389.064190] LustreError: 357943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12390.896029] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12390.908978] Lustre: Skipped 1 previous similar message [12390.917155] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12395.625134] Lustre: server umount lustre-MDT0001 complete [12400.596570] Lustre: server umount lustre-OST0000 complete [12412.089178] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [12425.104543] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [12461.366952] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 10:06:17 (1787407577) [12468.061384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12468.364990] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12468.408378] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12468.517493] Lustre: lustre-MDT0000: new disk, initializing [12468.612914] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12468.626334] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12474.324911] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [12487.587690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12487.735339] Lustre: 382103:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12487.801643] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12487.809378] Lustre: Skipped 1 previous similar message [12487.891983] Lustre: lustre-MDT0001: new disk, initializing [12488.031399] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12488.110667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12488.127372] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12494.104686] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [12503.401845] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12509.346772] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12515.800455] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12516.096976] Lustre: lustre-OST0000: new disk, initializing [12516.101423] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12516.109603] Lustre: 384072:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12516.268311] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12521.615302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12521.631084] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12521.823945] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12523.383211] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_default_debug -1 all [12534.329353] Lustre: DEBUG MARKER: oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12543.372762] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12543.387320] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12543.400085] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12543.406559] Lustre: Skipped 1 previous similar message [12543.973927] Lustre: server umount lustre-MDT0000 complete [12547.557147] LustreError: 382114:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12547.594667] LustreError: 382114:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [12547.974129] LustreError: 382897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787407666 with bad export cookie 5005524566908659233 [12547.975501] LustreError: 382911:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12547.990798] LustreError: MGC192.168.203.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12548.374896] Lustre: server umount lustre-MDT0001 complete [12563.176935] Lustre: server umount lustre-OST0000 complete [12590.441938] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing set_hostid [12600.073977] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing load_modules_local [12632.225903] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12386 sec ========= 10:09:08 (1787407748) [12634.403924] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 10:09:10 (1787407750) === [12637.757802] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 10:09:14 (1787407754) === [12679.371951] Lustre: DEBUG MARKER: oleg336-server.virtnet: executing unload_modules_local [12683.650425] Key type lgssc unregistered [12684.014567] LNet: 390439:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12684.032459] LNetError: 390439:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12684.064177] LNet: Removed LNI 192.168.203.136@tcp [12685.294558] Key type .llcrypt unregistered [12685.296431] Key type ._llcrypt unregistered