[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 503828571 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524592K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002360] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004017] kvm-guest: setup PV IPIs [ 0.006861] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007028] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008016] pid_max: default: 32768 minimum: 301 [ 0.009145] LSM: Security Framework initializing [ 0.010091] Yama: becoming mindful. [ 0.011048] SELinux: Initializing. [ 0.012069] *** VALIDATE selinux *** [ 0.020806] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024817] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025193] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026143] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027148] *** VALIDATE tmpfs *** [ 0.029322] *** VALIDATE proc *** [ 0.030295] *** VALIDATE cgroup *** [ 0.031013] *** VALIDATE cgroup2 *** [ 0.033203] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034204] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037026] Spectre V2 : User space: Vulnerable [ 0.038011] Speculative Store Bypass: Vulnerable [ 0.041040] debug: unmapping init [mem 0xffffffff98c59000-0xffffffff98c60fff] [ 0.044000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044695] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045030] ... version: 2 [ 0.046023] ... bit width: 48 [ 0.047014] ... generic registers: 4 [ 0.048015] ... value mask: 0000ffffffffffff [ 0.049016] ... max period: 00007fffffffffff [ 0.050018] ... fixed-purpose events: 3 [ 0.051017] ... event mask: 000000070000000f [ 0.052306] rcu: Hierarchical SRCU implementation. [ 0.054238] smp: Bringing up secondary CPUs ... [ 0.055607] x86: Booting SMP configuration: [ 0.056037] .... node #0, CPUs: #1 #2 #3 [ 0.059678] smp: Brought up 1 node, 4 CPUs [ 0.061016] smpboot: Max logical packages: 1 [ 0.062029] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.136251] node 0 deferred pages initialised in 71ms [ 0.138183] devtmpfs: initialized [ 0.139311] x86/mm: Memory block size: 128MB [ 0.142710] gcov: version magic: 0x41383552 [ 0.144259] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.147116] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.150345] pinctrl core: initialized pinctrl subsystem [ 0.152241] [ 0.152800] ************************************************************* [ 0.154015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.157016] ** ** [ 0.159021] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.161012] ** ** [ 0.163017] ** This means that this kernel is built to expose internal ** [ 0.166014] ** IOMMU data structures, which may compromise security on ** [ 0.168013] ** your system. ** [ 0.170025] ** ** [ 0.173016] ** If you see this message and you are not debugging the ** [ 0.175013] ** kernel, report this immediately to your vendor! ** [ 0.179019] ** ** [ 0.182017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.185031] ************************************************************* [ 0.189156] NET: Registered protocol family 16 [ 0.191529] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.195108] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.199126] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.207055] cpuidle: using governor menu [ 0.209135] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.210270] PCI: Using configuration type 1 for base access [ 0.212154] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.222117] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.224080] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.227115] cryptd: max_cpu_qlen set to 1000 [ 0.229226] ACPI: Added _OSI(Module Device) [ 0.230013] ACPI: Added _OSI(Processor Device) [ 0.232015] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.233014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.238878] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.245454] ACPI: Interpreter enabled [ 0.246000] ACPI: PM: (supports S0 S3 S4 S5) [ 0.249015] ACPI: Using IOAPIC for interrupt routing [ 0.251126] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.256174] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.269663] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.272051] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.274021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.277106] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.282308] acpiphp: Slot [2] registered [ 0.284155] acpiphp: Slot [5] registered [ 0.286183] acpiphp: Slot [6] registered [ 0.288143] acpiphp: Slot [7] registered [ 0.289139] acpiphp: Slot [8] registered [ 0.291182] acpiphp: Slot [9] registered [ 0.293124] acpiphp: Slot [10] registered [ 0.294147] acpiphp: Slot [3] registered [ 0.296118] acpiphp: Slot [4] registered [ 0.297109] acpiphp: Slot [11] registered [ 0.299120] acpiphp: Slot [12] registered [ 0.300120] acpiphp: Slot [13] registered [ 0.302139] acpiphp: Slot [14] registered [ 0.303125] acpiphp: Slot [15] registered [ 0.305195] acpiphp: Slot [16] registered [ 0.307156] acpiphp: Slot [17] registered [ 0.308141] acpiphp: Slot [18] registered [ 0.310127] acpiphp: Slot [19] registered [ 0.311106] acpiphp: Slot [20] registered [ 0.313140] acpiphp: Slot [21] registered [ 0.314102] acpiphp: Slot [22] registered [ 0.316117] acpiphp: Slot [23] registered [ 0.318100] acpiphp: Slot [24] registered [ 0.319133] acpiphp: Slot [25] registered [ 0.321139] acpiphp: Slot [26] registered [ 0.322124] acpiphp: Slot [27] registered [ 0.324157] acpiphp: Slot [28] registered [ 0.326128] acpiphp: Slot [29] registered [ 0.327149] acpiphp: Slot [30] registered [ 0.329148] acpiphp: Slot [31] registered [ 0.330076] PCI host bridge to bus 0000:00 [ 0.332031] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.335043] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.337027] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.340027] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.343033] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.347033] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.349203] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.352195] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.356521] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.367922] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.374068] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.378029] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.382025] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.385026] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.389557] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.393764] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.396037] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.399867] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.405018] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.421020] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.428971] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.435602] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.445018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.452018] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.471300] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.484518] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.490015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.500016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.525014] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.540796] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.550018] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.557015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.576018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.585255] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.593017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.603019] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.623016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.634245] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.653017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.664023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.685016] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.698053] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.706016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.719015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.741022] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.756271] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.759355] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.760343] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.763493] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.765382] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.772116] iommu: Default domain type: Passthrough [ 0.773000] SCSI subsystem initialized [ 0.774197] ACPI: bus type USB registered [ 0.776118] usbcore: registered new interface driver usbfs [ 0.778099] usbcore: registered new interface driver hub [ 0.780145] usbcore: registered new device driver usb [ 0.782184] pps_core: LinuxPPS API ver. 1 registered [ 0.784015] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.787063] PTP clock support registered [ 0.789073] EDAC MC: Ver: 3.0.0 [ 0.790499] PCI: Using ACPI for IRQ routing [ 0.792925] NetLabel: Initializing [ 0.794016] NetLabel: domain hash size = 128 [ 0.796012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.798092] NetLabel: unlabeled traffic allowed by default [ 0.801125] vgaarb: loaded [ 0.803248] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.804018] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.811004] clocksource: Switched to clocksource kvm-clock [ 0.946199] VFS: Disk quotas dquot_6.6.0 [ 0.948426] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.951623] *** VALIDATE ramfs *** [ 0.953277] *** VALIDATE hugetlbfs *** [ 0.955936] pnp: PnP ACPI init [ 0.958583] pnp: PnP ACPI: found 6 devices [ 0.979549] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.983204] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.985604] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.988430] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.991134] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.993836] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.996793] NET: Registered protocol family 2 [ 0.999373] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.004230] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.008874] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.014488] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.017995] TCP: Hash tables configured (established 65536 bind 65536) [ 1.021156] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.024526] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.026857] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.029949] NET: Registered protocol family 1 [ 1.032498] RPC: Registered named UNIX socket transport module. [ 1.034485] RPC: Registered udp transport module. [ 1.039376] RPC: Registered tcp transport module. [ 1.041397] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.044237] NET: Registered protocol family 44 [ 1.045960] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.048745] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.051917] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.055080] PCI: CLS 0 bytes, default 64 [ 1.058161] Unpacking initramfs... [ 2.479527] debug: unmapping init [mem 0xffff9db83cc54000-0xffff9db83ffbffff] [ 2.484255] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.486880] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.489840] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.010812] Initialise system trusted keyrings [ 3.012413] Key type blacklist registered [ 3.014224] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.033089] zbud: loaded [ 3.036789] *** VALIDATE nfs *** [ 3.038275] *** VALIDATE nfs4 *** [ 3.040107] pstore: using deflate compression [ 3.044481] Platform Keyring initialized [ 3.153205] NET: Registered protocol family 38 [ 3.155610] Key type asymmetric registered [ 3.159389] Asymmetric key parser 'x509' registered [ 3.161950] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.165236] io scheduler mq-deadline registered [ 3.166684] io scheduler kyber registered [ 3.169117] io scheduler bfq registered [ 3.171516] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.175485] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.179001] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.182474] ACPI: Power Button [PWRF] [ 3.188747] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.198375] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.214983] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.222423] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.244817] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.272789] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.302053] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.309635] Non-volatile memory driver v1.3 [ 3.311493] Linux agpgart interface v0.103 [ 3.345561] virtio_blk virtio1: [vda] 146152 512-byte logical blocks (74.8 MB/71.4 MiB) [ 3.349066] vda: detected capacity change from 0 to 74829824 [ 3.365761] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.369402] vdb: detected capacity change from 0 to 1073741824 [ 3.387370] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.390663] vdc: detected capacity change from 0 to 2621440000 [ 3.409488] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.412564] vdd: detected capacity change from 0 to 2621440000 [ 3.428895] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.432247] vde: detected capacity change from 0 to 4294967296 [ 3.450146] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.453835] vdf: detected capacity change from 0 to 4294967296 [ 3.462945] libphy: Fixed MDIO Bus: probed [ 3.468192] usbcore: registered new interface driver usbserial_generic [ 3.471117] usbserial: USB Serial support registered for generic [ 3.473745] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.478223] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.479962] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.482383] mousedev: PS/2 mouse device common for all mice [ 3.485262] rtc_cmos 00:05: RTC can wake from S4 [ 3.488304] rtc_cmos 00:05: registered as rtc0 [ 3.490339] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.493165] intel_pstate: CPU model not supported [ 3.495454] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.502099] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.502581] hid: raw HID events driver (C) Jiri Kosina [ 3.507254] usbcore: registered new interface driver usbhid [ 3.509260] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.509810] usbhid: USB HID core driver [ 3.515193] drop_monitor: Initializing network drop monitor service [ 3.517753] Initializing XFRM netlink socket [ 3.519465] NET: Registered protocol family 10 [ 3.522142] Segment Routing with IPv6 [ 3.523688] NET: Registered protocol family 17 [ 3.525804] mpls_gso: MPLS GSO support [ 3.531741] RAS: Correctable Errors collector initialized. [ 3.534022] AVX version of gcm_enc/dec engaged. [ 3.536152] AES CTR mode by8 optimization enabled [ 3.610713] sched_clock: Marking stable (3610675261, 0)->(4534753019, -924077758) [ 3.614238] registered taskstats version 1 [ 3.616233] Loading compiled-in X.509 certificates [ 3.617798] zswap: loaded using pool lzo/zbud [ 3.644818] Key type big_key registered [ 3.657085] Key type encrypted registered [ 3.659408] ima: No TPM chip found, activating TPM-bypass! [ 3.661840] ima: Allocated hash algorithm: sha1 [ 3.663764] ima: No architecture policies found [ 3.665849] evm: Initialising EVM extended attributes: [ 3.668013] evm: security.selinux [ 3.669253] evm: security.ima [ 3.670329] evm: security.capability [ 3.671749] evm: HMAC attrs: 0x1 [ 3.674353] rtc_cmos 00:05: setting system clock to 2026-08-24 14:29:14 UTC (1787581754) [ 3.682252] debug: unmapping init [mem 0xffffffff99c03000-0xffffffff99dfffff] [ 3.686514] debug: unmapping init [mem 0xffffffff98982000-0xffffffff98c58fff] [ 3.696107] Write protecting the kernel read-only data: 28672k [ 3.699922] debug: unmapping init [mem 0xffffffff97003000-0xffffffff971fffff] [ 3.703734] debug: unmapping init [mem 0xffffffff97914000-0xffffffff979fffff] [ 3.747465] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.756671] systemd[1]: Detected virtualization kvm. [ 3.758922] systemd[1]: Detected architecture x86-64. [ 3.761429] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.788635] systemd[1]: No hostname configured. [ 3.790908] systemd[1]: Set hostname to . [ 3.793651] random: systemd: uninitialized urandom read (16 bytes read) [ 3.797912] systemd[1]: Initializing machine ID from random generator. [ 3.929238] random: systemd: uninitialized urandom read (16 bytes read) [ 3.933217] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.937648] random: systemd: uninitialized urandom read (16 bytes read) [ 3.941458] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.947330] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Initrd Root Device. Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.643246] device-mapper: uevent: version 1.0.3 [ 4.645702] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.498181] random: fast init done [ 5.540653] virtio_net virtio0 ens2: renamed from eth0 [ 5.547114] scsi host0: ata_piix [ 5.577894] scsi host1: ata_piix [ 5.580093] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.584322] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.053271] dracut-initqueue[586]: RTNETLINK answers: File exists [ 10.177810] random: crng init done [ 10.180773] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.767267] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.008880] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.317490] SELinux: Disabled at runtime. [ 12.379616] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.389516] systemd[1]: Detected virtualization kvm. [ 12.391339] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.942579] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.946739] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.952068] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.958395] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.962318] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.969898] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.978770] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Remount Root and Kernel File Systems... Mounting Huge Pages File System... Mounting Kernel Debug File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. [ OK ] Created slice system-getty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Control Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. [ 13.142128] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting udev Coldplug all Devices... [ OK ] Reached target RPC Port Mapper. Mounting POSIX Message Queue File System... [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.483062] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.736032] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.781411] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.904273] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.915898] EDAC sbridge: Ver: 1.1.2 [ 15.681854] Key type dns_resolver registered [ 15.985337] NFS: Registering the id_resolver key type [ 15.988062] Key type id_resolver registered [ 15.989874] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. [ OK ] Started Authorization Manager. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg431-server login: [ 74.556703] libcfs: loading out-of-tree module taints kernel. [ 74.592603] Key type ._llcrypt registered [ 74.596160] Key type .llcrypt registered [ 74.767992] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 92.065337] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 93.859986] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 93.878629] alg: No test for adler32 (adler32-zlib) [ 95.531275] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 96.613924] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 98.495171] Key type lgssc registered [ 100.591204] Lustre: Echo OBD driver; http://www.lustre.org/ [ 102.243079] hrtimer: interrupt took 4907320 ns [ 117.517674] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 165.798630] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 181.542608] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 181.580312] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 182.844641] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 182.898356] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 183.087829] Lustre: lustre-MDT0000: new disk, initializing [ 183.266729] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 183.294461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 187.731639] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 203.158892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 203.273253] Lustre: 6503:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 203.320917] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 203.328368] Lustre: Skipped 1 previous similar message [ 203.398808] Lustre: lustre-MDT0001: new disk, initializing [ 203.480542] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 203.517684] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 203.534175] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 208.248885] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 213.220868] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 224.013907] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 224.473708] Lustre: lustre-OST0000: new disk, initializing [ 224.479979] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 224.482046] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 224.491658] Lustre: 8443:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 224.589864] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 229.892677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 229.913520] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 230.007188] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 231.971790] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 248.192576] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 248.310210] Lustre: lustre-OST0001: new disk, initializing [ 248.313461] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 248.321819] Lustre: 9515:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 248.389313] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 255.239508] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 256.042211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 256.059365] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 256.099741] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 265.689823] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 271.662950] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 277.638705] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing check_logdir /tmp/testlogs/ [ 283.518687] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing yml_node [ 288.100983] Lustre: DEBUG MARKER: Client: 2.17.57.80 [ 290.502839] Lustre: DEBUG MARKER: MDS: 2.17.57.80 [ 293.050527] Lustre: DEBUG MARKER: OSS: 2.17.57.80 [ 294.643717] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 24 10:34:02 EDT 2026 [ 314.383155] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 315.712915] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 331.749412] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 331.766445] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 331.781118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 332.768465] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 332.787804] Lustre: Skipped 2 previous similar messages [ 332.792757] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 332.794989] Lustre: Skipped 3 previous similar messages [ 337.035968] Lustre: server umount lustre-MDT0000 complete [ 340.999076] LustreError: 6496:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787582091 with bad export cookie 10861759911034077643 [ 341.003971] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 341.015668] LustreError: 6496:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 343.008079] LustreError: 6509:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 343.008270] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 343.008589] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 343.026603] LustreError: 6509:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 343.070184] Lustre: Skipped 1 previous similar message [ 346.016707] LustreError: 6510:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 346.042617] LustreError: 6510:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 347.428302] Lustre: server umount lustre-MDT0001 complete [ 351.224764] Lustre: server umount lustre-OST0000 complete [ 354.959614] Lustre: server umount lustre-OST0001 complete [ 362.273423] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 373.088744] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 411.001658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 411.290770] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 411.323681] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 411.431606] Lustre: lustre-MDT0000: new disk, initializing [ 411.487723] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 411.514854] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 416.646463] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 428.634329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 428.774524] Lustre: 15065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 428.838506] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 428.842956] Lustre: Skipped 1 previous similar message [ 428.923638] Lustre: lustre-MDT0001: new disk, initializing [ 429.011536] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 429.059121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 429.071794] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 433.801733] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 443.465839] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 449.725212] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 456.820589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 457.120989] Lustre: lustre-OST0000: new disk, initializing [ 457.125779] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 457.136409] Lustre: 17035:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 457.243479] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 458.850140] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 458.871212] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 458.976028] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 464.093438] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 475.107081] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 480.236760] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 480.441035] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 484.987602] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 485.242893] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 489.952498] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 489.964816] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 489.975369] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 489.982675] Lustre: Skipped 1 previous similar message [ 490.000313] Lustre: Skipped 3 previous similar messages [ 493.519590] Lustre: server umount lustre-OST0000 complete [ 500.197909] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 500.228232] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 500.244297] Lustre: Skipped 2 previous similar messages [ 504.396330] Lustre: server umount lustre-MDT0000 complete [ 505.312334] LustreError: 15074:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 505.337504] LustreError: 15074:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 508.777656] LustreError: 15058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787582259 with bad export cookie 10861759911034079176 [ 508.780655] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 508.786245] LustreError: 15058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 509.294732] Lustre: server umount lustre-MDT0001 complete [ 515.065082] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:37:43 (1787582263) [ 522.187329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 522.833813] LustreError: 19301:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 522.920468] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 527.860837] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 528.355561] LustreError: 19302:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 537.569624] LustreError: 19301:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 537.615144] LustreError: 19301:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 537.784926] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 546.036460] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 557.225532] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 565.018816] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 572.651771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 580.666643] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 589.892611] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 598.952566] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 608.228528] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 608.240921] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 608.248597] Lustre: Skipped 2 previous similar messages [ 608.268643] Lustre: Skipped 2 previous similar messages [ 612.746430] Lustre: server umount lustre-MDT0000 complete [ 613.351861] LustreError: 19296:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 613.377970] LustreError: 19296:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 616.775351] LustreError: 19282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787582367 with bad export cookie 10861759911034080058 [ 616.781434] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 616.787313] LustreError: 19282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 617.205672] Lustre: server umount lustre-MDT0001 complete [ 632.342682] Lustre: server umount lustre-OST0000 complete [ 657.199855] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 670.309519] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 670.743991] LustreError: 23685:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 670.812907] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 670.820292] Lustre: Skipped 2 previous similar messages [ 676.404417] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 689.563733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 695.779642] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 698.864108] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 707.115454] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 715.003402] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 725.421283] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 725.665450] Lustre: lustre-OST0001: new disk, initializing [ 725.678965] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 725.704982] Lustre: 26004:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 727.509171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 727.531663] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 727.624705] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 732.907991] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 743.980352] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 748.672620] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 763.364468] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 763.369115] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 763.386382] Lustre: Skipped 3 previous similar messages [ 763.414073] Lustre: Skipped 3 previous similar messages [ 768.955878] Lustre: server umount lustre-MDT0000 complete [ 772.750243] LustreError: 24427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787582523 with bad export cookie 10861759911034080905 [ 772.763599] LustreError: 24427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 772.763692] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 773.068745] Lustre: server umount lustre-MDT0001 complete [ 786.632396] Lustre: server umount lustre-OST0000 complete [ 789.984296] Lustre: 3637:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787582524/real 1787582524] req@ffff9db8bf88c000 x1874415423048192/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787582540 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 791.039046] Lustre: server umount lustre-OST0001 complete [ 810.209920] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 820.973043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 821.342179] LustreError: 29022:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 821.356723] LustreError: 29022:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 821.466936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 821.476622] Lustre: Skipped 3 previous similar messages [ 826.265729] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 835.270769] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 840.875582] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 843.825788] Lustre: 30171:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 849.999048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 856.765245] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 866.378385] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 872.808858] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 882.222958] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 886.054682] Lustre: 32051:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 893.412923] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 893.421946] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 893.434242] Lustre: Skipped 5 previous similar messages [ 893.450914] Lustre: Skipped 7 previous similar messages [ 898.409843] Lustre: server umount lustre-MDT0000 complete [ 902.683705] LustreError: 29004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787582653 with bad export cookie 10861759911034082200 [ 902.701247] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 903.028708] Lustre: server umount lustre-MDT0001 complete [ 917.302026] Lustre: server umount lustre-OST0000 complete [ 919.523908] Lustre: 3639:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787582654/real 1787582654] req@ffff9db8bb165180 x1874415423098496/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787582670 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 921.363995] Lustre: server umount lustre-OST0001 complete [ 932.360876] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 10:44:40 (1787582680) [ 966.616320] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 978.096821] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 978.519252] LustreError: 34854:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 978.533600] LustreError: 34854:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 983.963491] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 994.759764] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1000.235422] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1003.668778] Lustre: 36006:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1011.533881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1019.113185] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1028.131872] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1035.199832] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1043.790829] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1047.152619] Lustre: 37885:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1051.621670] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1051.627155] LustreError: Skipped 1 previous similar message [ 1051.633374] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1051.645930] Lustre: Skipped 1 previous similar message [ 1051.650271] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1057.909588] Lustre: server umount lustre-MDT0000 complete [ 1062.212712] LustreError: 35609:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787582813 with bad export cookie 10861759911034083306 [ 1062.221358] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1062.227876] LustreError: 35609:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1062.844214] Lustre: server umount lustre-MDT0001 complete [ 1077.902249] Lustre: server umount lustre-OST0000 complete [ 1080.799350] Lustre: 3636:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787582815/real 1787582815] req@ffff9db8b6350000 x1874415423149312/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787582831 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1082.312031] Lustre: server umount lustre-OST0001 complete [ 1093.093725] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 10:47:20 (1787582840) [ 1115.528542] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1127.750226] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1128.368290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1128.380571] Lustre: Skipped 7 previous similar messages [ 1134.312558] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1145.130831] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1150.762784] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1154.281446] Lustre: 41191:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1161.837640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1170.380575] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1179.760935] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1187.403482] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1196.029977] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1199.916507] Lustre: 43072:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1200.897945] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1200.902517] Lustre: Skipped 1 previous similar message [ 1205.740441] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1205.757114] Lustre: Skipped 4 previous similar messages [ 1210.885534] Lustre: server umount lustre-MDT0000 complete [ 1215.465331] LustreError: 40023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787582966 with bad export cookie 10861759911034084412 [ 1215.468747] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1215.495518] LustreError: 40023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1221.798572] Lustre: server umount lustre-MDT0001 complete [ 1230.503617] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1240.555744] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x96bcb9174cc3e0a3 [ 1240.575971] Lustre: MGC192.168.204.131@tcp: Connection restored to 0@lo (at 0@lo) [ 1240.852603] LustreError: 41559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1240.886156] LustreError: 41559:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 30 previous similar messages [ 1246.088897] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1250.226253] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1254.777598] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1260.341625] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1260.537074] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1260.548906] Lustre: Skipped 1 previous similar message [ 1268.384473] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1269.723710] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1275.058614] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1276.512889] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1286.132505] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1288.458849] Lustre: server umount lustre-MDT0000 complete [ 1292.007483] LustreError: 40023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787583042 with bad export cookie 10861759911034085539 [ 1292.013033] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1292.032267] LustreError: 40023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1292.318632] Lustre: server umount lustre-MDT0001 complete [ 1305.648355] Lustre: server umount lustre-OST0000 complete [ 1320.517777] Lustre: server umount lustre-OST0001 complete [ 1329.095754] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 10:51:17 (1787583077) [ 1345.625587] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1355.201046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1359.696372] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1368.654899] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1374.029037] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1377.328862] Lustre: 48838:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1386.613748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1395.582311] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1405.254703] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1413.548459] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1422.044346] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1427.072853] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1440.741513] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1440.752180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1440.757390] Lustre: Skipped 19 previous similar messages [ 1441.977128] Lustre: server umount lustre-MDT0000 complete [ 1445.511544] LustreError: 47665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787583196 with bad export cookie 10861759911034086330 [ 1445.520242] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1445.523591] LustreError: 47665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1445.828308] Lustre: server umount lustre-MDT0001 complete [ 1459.520342] Lustre: server umount lustre-OST0000 complete [ 1462.431759] Lustre: 3637:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787583197/real 1787583197] req@ffff9db782a54380 x1874415423298432/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787583213 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1462.459526] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1462.470165] Lustre: Skipped 12 previous similar messages [ 1463.706965] Lustre: server umount lustre-OST0001 complete [ 1475.015932] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 10:53:43 (1787583223) [ 1478.348163] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1480.655780] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 10:53:48 (1787583228) [ 1494.036930] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 1505.755699] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1531.210695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1531.545912] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1531.632632] Lustre: lustre-MDT0000: new disk, initializing [ 1531.753210] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1537.274100] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1548.316362] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1556.489175] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1557.001796] Lustre: lustre-OST0000: new disk, initializing [ 1557.007169] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1557.016744] Lustre: Skipped 1 previous similar message [ 1557.027260] Lustre: 54913:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1558.378542] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1558.396188] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1558.443434] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1565.377875] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1577.563273] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1580.323422] Lustre: DEBUG MARKER: create test files [ 1619.948694] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1621.544252] Lustre: server umount lustre-OST0000 complete [ 1627.782788] Lustre: server umount lustre-MDT0000 complete [ 1643.221546] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 1648.269404] Key type lgssc unregistered [ 1648.863244] LNet: 56550:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1648.890366] LNetError: 56550:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1648.929483] LNet: Removed LNI 192.168.204.131@tcp [ 1650.660242] Key type .llcrypt unregistered [ 1650.667251] Key type ._llcrypt unregistered [ 1682.957143] Key type ._llcrypt registered [ 1682.960186] Key type .llcrypt registered [ 1683.080376] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1684.770338] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1684.847456] alg: No test for adler32 (adler32-zlib) [ 1685.890654] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 1686.153794] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 1687.839183] Key type lgssc registered [ 1689.288857] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1697.934965] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1697.959051] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1699.679477] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1704.800789] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1713.693982] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1721.897824] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1722.350923] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1724.400293] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1724.485099] Lustre: 59288:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1724.496955] Lustre: 59288:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1724.506220] Lustre: 59288:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1724.517134] Lustre: 59288:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1724.522508] Lustre: 59288:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1724.530745] Lustre: 59288:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1728.796460] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1736.394706] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1741.851467] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1747.622909] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1755.106933] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1755.118994] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1755.132692] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1757.285740] Lustre: server umount lustre-OST0000 complete [ 1761.633957] Lustre: server umount lustre-MDT0000 complete [ 1774.062726] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 1777.842225] Key type lgssc unregistered [ 1778.228729] LNet: 60898:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1778.249997] LNetError: 60898:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1778.275950] LNet: Removed LNI 192.168.204.131@tcp [ 1779.580116] Key type .llcrypt unregistered [ 1779.583324] Key type ._llcrypt unregistered [ 1809.603894] Key type ._llcrypt registered [ 1809.609547] Key type .llcrypt registered [ 1809.758750] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1811.071392] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1811.158408] alg: No test for adler32 (adler32-zlib) [ 1812.177178] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 1812.471463] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 1814.151238] Key type lgssc registered [ 1815.242706] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1822.690360] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1822.725044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1824.204364] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1829.391960] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1837.531832] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1845.161863] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1845.563747] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1848.651097] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1848.721464] Lustre: 63731:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1848.730179] Lustre: 63731:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1848.739091] Lustre: 63731:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1848.745971] Lustre: 63731:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1848.754981] Lustre: 63731:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1848.764293] Lustre: 63731:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1852.205458] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1861.426295] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1866.734429] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1909.228977] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1909.261877] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1911.648661] Lustre: server umount lustre-OST0000 complete [ 1916.155524] Lustre: server umount lustre-MDT0000 complete [ 1930.395643] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 1934.130114] Key type lgssc unregistered [ 1934.402200] LNet: 65330:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1934.416062] LNetError: 65330:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1934.440639] LNet: Removed LNI 192.168.204.131@tcp [ 1935.481041] Key type .llcrypt unregistered [ 1935.484106] Key type ._llcrypt unregistered [ 1953.426785] Key type ._llcrypt registered [ 1953.429020] Key type .llcrypt registered [ 1953.509351] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 1969.372357] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1970.945643] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1971.031261] alg: No test for adler32 (adler32-zlib) [ 1972.030625] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 1972.307308] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 1973.983211] Key type lgssc registered [ 1975.304341] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2013.569120] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2013.631069] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2014.988426] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2015.044590] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2015.146948] Lustre: lustre-MDT0000: new disk, initializing [ 2015.244228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2015.266380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2020.588138] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2032.243084] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2032.370223] Lustre: 68484:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2032.442272] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2032.446111] Lustre: Skipped 1 previous similar message [ 2032.516437] Lustre: lustre-MDT0001: new disk, initializing [ 2032.620189] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2032.664737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2032.678178] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2036.667850] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2045.384949] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2051.086938] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2057.260810] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2057.507709] Lustre: lustre-OST0000: new disk, initializing [ 2057.515598] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2057.526593] Lustre: 70450:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2057.659704] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2063.659588] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2067.032380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2067.053430] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2067.240876] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2074.429279] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2079.405846] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2079.652676] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2084.622421] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2084.785397] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2087.916506] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2087.918997] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2087.932118] LustreError: Skipped 1 previous similar message [ 2087.935356] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2087.943346] Lustre: Skipped 1 previous similar message [ 2092.846369] Lustre: server umount lustre-OST0000 complete [ 2098.146516] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2098.148945] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2098.163421] Lustre: Skipped 1 previous similar message [ 2098.174401] Lustre: Skipped 1 previous similar message [ 2102.988653] Lustre: server umount lustre-MDT0000 complete [ 2107.291484] LustreError: 69279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787583858 with bad export cookie 16068947522018914285 [ 2107.294292] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2107.306097] LustreError: 69279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2107.668546] Lustre: server umount lustre-MDT0001 complete [ 2117.588351] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 11:04:25 (1787583865) [ 2131.475528] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 2173.324729] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 2183.596917] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 2231.051849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2231.240688] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2231.263359] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2231.315567] Lustre: lustre-MDT0000: new disk, initializing [ 2231.408454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2231.448093] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2237.838658] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2250.336686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2250.427773] Lustre: 76487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2250.479896] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2250.486692] Lustre: Skipped 1 previous similar message [ 2250.544577] Lustre: lustre-MDT0001: new disk, initializing [ 2250.690662] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2250.801424] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2250.816170] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2255.611946] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2265.827558] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2272.178585] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2280.941706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2281.413692] Lustre: lustre-OST0000: new disk, initializing [ 2281.423375] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2281.431923] Lustre: 78458:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2281.580308] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2288.199222] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2288.209146] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2288.372024] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2288.393043] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2298.435258] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2302.915066] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2303.183425] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2307.300085] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2307.612935] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2314.208348] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2314.210306] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2314.213976] LustreError: Skipped 1 previous similar message [ 2314.225324] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2314.237027] Lustre: Skipped 1 previous similar message [ 2315.597611] Lustre: server umount lustre-OST0000 complete [ 2319.328742] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2319.336457] Lustre: Skipped 1 previous similar message [ 2319.339826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2319.342901] Lustre: Skipped 1 previous similar message [ 2324.448751] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2324.457241] Lustre: Skipped 1 previous similar message [ 2325.438623] Lustre: server umount lustre-MDT0000 complete [ 2329.321469] LustreError: 77282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787584080 with bad export cookie 16068947522018915167 [ 2329.322337] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2329.332253] LustreError: 77282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2329.712671] Lustre: server umount lustre-MDT0001 complete [ 2339.046365] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 11:08:07 (1787584087) [ 2346.697868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2347.153396] LustreError: 80880:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2347.187396] LustreError: 80880:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2347.289021] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2352.635015] LustreError: 80881:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2353.154896] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2357.729401] LustreError: 80880:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2362.860897] LustreError: 80881:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2365.098709] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2370.389924] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2378.575745] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2384.919380] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2392.106189] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2392.623733] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2392.635253] Lustre: Skipped 1 previous similar message [ 2399.623517] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2407.394992] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2411.615367] Lustre: 83554:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 53048023-abf7-4fe2-83d8-8e5d1a6d1302 at adminstrative request [ 2416.614353] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2416.644562] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2426.855410] Lustre: MGS: Client 53048023-abf7-4fe2-83d8-8e5d1a6d1302 (at 0@lo) reconnecting [ 2426.866439] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xdf005ea26454b6d1 to 0xdf005ea26454b971 [ 2426.892780] Lustre: MGC192.168.204.131@tcp: Connection restored to 0@lo (at 0@lo) [ 2449.298318] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2449.522463] Lustre: lustre-OST0001: new disk, initializing [ 2449.526500] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2449.542199] Lustre: 83949:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2456.658516] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2456.667440] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2456.771399] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2456.935389] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2468.058441] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2472.423730] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2472.437936] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2472.445364] Lustre: Skipped 2 previous similar messages [ 2472.447354] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2472.455204] Lustre: Skipped 1 previous similar message [ 2477.649390] Lustre: server umount lustre-OST0001 complete [ 2482.661397] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2482.673083] LustreError: 82705:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2482.680595] Lustre: Skipped 1 previous similar message [ 2482.706552] LustreError: 82705:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2488.055288] Lustre: server umount lustre-OST0000 complete [ 2492.900388] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2492.905685] Lustre: Skipped 7 previous similar messages [ 2506.207774] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2506.497255] Lustre: server umount lustre-MDT0000 complete [ 2508.260797] LustreError: 81609:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2508.272378] LustreError: 81609:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2510.152130] LustreError: 80860:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787584260 with bad export cookie 16068947522018916721 [ 2510.160594] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2510.189298] LustreError: 80860:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2510.575537] Lustre: server umount lustre-MDT0001 complete [ 2520.055738] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 11:11:08 (1787584268) [ 2522.127530] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2524.133746] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 11:11:12 (1787584272) [ 2525.840848] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2528.136188] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 11:11:16 (1787584276) [ 2530.075396] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2532.577625] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 11:11:20 (1787584280) [ 2534.461564] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2536.664483] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 11:11:24 (1787584284) [ 2538.389584] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2545.813163] loop: module loaded [ 2545.857961] loop0: detected capacity change from 0 to 4294967296 [ 2545.941240] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2548.937802] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2554.249514] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2562.139037] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2595.704859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2595.753589] LustreError: 88076:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2595.759317] LustreError: 88076:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2595.764766] LustreError: 88076:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2595.770469] LustreError: 88076:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2617.224600] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 2629.421152] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 2669.527923] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2669.779967] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2669.803575] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2669.887653] Lustre: lustre-MDT0000: new disk, initializing [ 2669.963833] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2669.968328] Lustre: Skipped 1 previous similar message [ 2669.987345] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2675.083809] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2687.143388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2687.306321] Lustre: 90942:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2687.571323] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2692.933093] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2705.177540] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2712.802392] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2721.126968] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2721.485534] Lustre: 92912:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2728.030601] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2728.198966] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2729.078950] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2741.584598] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2747.046449] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2747.379653] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2753.029750] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2753.229941] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2759.137898] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2759.138686] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2759.147205] LustreError: Skipped 1 previous similar message [ 2759.150126] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2759.150133] Lustre: Skipped 4 previous similar messages [ 2759.211371] Lustre: Skipped 3 previous similar messages [ 2761.480589] Lustre: server umount lustre-OST0000 complete [ 2772.219240] Lustre: server umount lustre-MDT0000 complete [ 2774.502420] LustreError: 90954:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2774.530143] LustreError: 90954:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2776.389497] LustreError: 90934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787584527 with bad export cookie 16068947522018917337 [ 2776.392704] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2776.413479] LustreError: 90934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2776.855751] Lustre: server umount lustre-MDT0001 complete [ 2787.474502] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 11:15:35 (1787584535) [ 2795.486489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2802.565288] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2810.337770] LustreError: 95327:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2810.361518] LustreError: 95327:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2813.829895] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2814.381855] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2814.394082] Lustre: Skipped 3 previous similar messages [ 2819.741360] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2829.647356] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2837.902834] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2846.602206] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2855.221187] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2865.639625] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2871.343614] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2871.689150] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2876.685563] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2876.930075] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2884.315831] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2884.484817] Lustre: lustre-OST0001: new disk, initializing [ 2884.489939] Lustre: Skipped 2 previous similar messages [ 2884.492891] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2884.503507] Lustre: Skipped 3 previous similar messages [ 2884.509594] Lustre: 98697:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2891.421226] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2891.435749] Lustre: Skipped 2 previous similar messages [ 2891.464058] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2891.570664] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2892.449687] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2903.853649] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2909.009505] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2909.237608] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2915.253287] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2915.591221] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2938.743443] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2968.845414] Lustre: Failing over lustre-MDT0000 [ 2969.704518] Lustre: server umount lustre-MDT0000 complete [ 2970.347113] LustreError: 95321:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.31@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2970.375065] LustreError: 95321:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2973.157349] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2973.179544] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2973.203932] Lustre: Skipped 2 previous similar messages [ 2990.111680] Lustre: 66708:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787584724/real 1787584724] req@000000000620bc23 x1874417390214912/t0(0) o400->MGC192.168.204.131@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787584740 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2990.141645] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2997.941271] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2997.945143] LDISKFS-fs (dm-0): recovery complete [ 2997.956199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2999.264835] LustreError: 66706:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000f0ac69f5 x1874417390223360/t0(0) o250->MGC192.168.204.131@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2999.620772] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3002.023144] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3002.024664] Lustre: 98705:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3004.825729] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3004.907949] Lustre: 95327:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3004.928300] Lustre: 95327:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 3004.944289] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 3005.045328] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3005.367295] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3005.700764] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3005.716163] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 3006.050478] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3006.063134] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 3006.080243] Lustre: 101550:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3006.104252] Lustre: 101550:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 3006.753758] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3006.766655] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 3007.087237] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3007.102628] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 3008.104351] Lustre: 101550:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3008.123443] Lustre: 101550:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 3008.769287] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3008.786030] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 3009.119180] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3009.127388] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 3012.351286] Lustre: 101550:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3012.368603] Lustre: 101550:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 3013.036575] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3013.049041] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 3013.383188] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3013.398184] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 3020.464329] Lustre: 101550:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3020.479289] Lustre: 101550:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 3021.130144] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3021.147073] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 3021.472134] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3021.486948] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 3036.693069] Lustre: 101550:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3036.706018] Lustre: 101550:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 3037.359753] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3037.379824] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3037.719201] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3037.736751] LustreError: 101550:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3062.239496] Lustre: 101550:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3062.259379] Lustre: 101550:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3062.281043] Lustre: 101550:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 4c35904b-5a3a-4941-b97c-1dc64b4ccc02@192.168.204.31@tcp [ 3062.291861] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3062.299844] Lustre: 101550:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000341df399 x1874417372173312/t0(8589934775) o101->4c35904b-5a3a-4941-b97c-1dc64b4ccc02@192.168.204.31@tcp:13/0 lens 592/0 e 0 to 0 dl 1787584823 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3062.336964] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3062.354154] Lustre: lustre-MDT0000: Denying connection for new client 4c35904b-5a3a-4941-b97c-1dc64b4ccc02 (at 192.168.204.31@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3062.400752] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3062.451492] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3062.456186] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3062.463653] Lustre: Skipped 2 previous similar messages [ 3062.503642] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 3062.506501] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3074.531839] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3074.536753] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3074.550588] Lustre: Skipped 3 previous similar messages [ 3074.556646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3074.564699] Lustre: Skipped 3 previous similar messages [ 3080.409744] Lustre: server umount lustre-OST0000 complete [ 3098.079125] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3098.295942] Lustre: server umount lustre-OST0001 complete [ 3103.718472] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3103.732062] LustreError: Skipped 1 previous similar message [ 3108.874849] Lustre: server umount lustre-MDT0000 complete [ 3112.423506] LustreError: 95326:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3112.446817] LustreError: 95326:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 49 previous similar messages [ 3112.552183] LustreError: 95307:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787584863 with bad export cookie 16068947522018968549 [ 3112.552615] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3112.564873] LustreError: 95307:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3112.909724] Lustre: server umount lustre-MDT0001 complete [ 3121.762428] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 11:21:09 (1787584869) [ 3130.857516] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3130.862821] Lustre: *** cfs_fail_loc=197, val=0*** [ 3146.465306] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3151.584850] LustreError: 103525:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.131@tcp: failed processing log, type 4: rc = -110 [ 3156.812689] Lustre: *** cfs_fail_loc=197, val=0*** [ 3156.819817] Lustre: Skipped 65 previous similar messages [ 3177.376202] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3177.385027] Lustre: Skipped 3 previous similar messages [ 3184.587873] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3193.764620] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3195.976222] Lustre: server umount lustre-OST0000 complete [ 3206.928145] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 11:22:35 (1787584955) [ 3229.051281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3234.270567] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3243.862168] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3248.531570] LustreError: 105175:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3248.546837] LustreError: 105175:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3248.559540] LustreError: 105175:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3249.050658] Lustre: server umount lustre-MDT0000 complete [ 3289.523753] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 3302.133992] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 3342.540961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3342.846335] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3342.871983] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3342.937432] Lustre: lustre-MDT0000: new disk, initializing [ 3343.009962] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3348.379912] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3360.789422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3360.925397] Lustre: 109354:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3361.280293] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3366.366550] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3377.983305] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3385.552829] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3394.636430] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3395.173691] Lustre: 111322:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3402.823929] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3403.598641] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3416.033439] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3421.207120] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3421.489851] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3427.857155] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3428.145719] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3434.470977] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3434.472726] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3434.482235] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3434.482246] Lustre: Skipped 11 previous similar messages [ 3434.505586] Lustre: Skipped 6 previous similar messages [ 3436.388133] Lustre: server umount lustre-OST0000 complete [ 3446.798467] Lustre: server umount lustre-MDT0000 complete [ 3449.841310] LustreError: 109360:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3449.872082] LustreError: 109360:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 3450.655110] LustreError: 109346:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787585201 with bad export cookie 16068947522018979140 [ 3450.664048] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3450.668687] LustreError: 109346:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3451.095882] Lustre: server umount lustre-MDT0001 complete [ 3461.533672] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 11:26:49 (1787585209) [ 3471.814871] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 3474.982868] Key type lgssc unregistered [ 3475.321178] LNet: 113812:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3475.335583] LNetError: 113812:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3475.356207] LNet: Removed LNI 192.168.204.131@tcp [ 3476.167481] Key type .llcrypt unregistered [ 3476.169431] Key type ._llcrypt unregistered [ 3495.826225] Key type ._llcrypt registered [ 3495.828434] Key type .llcrypt registered [ 3495.972447] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 3517.235727] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 3518.604900] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3518.885952] alg: No test for adler32 (adler32-zlib) [ 3520.076195] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 3520.412549] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 3522.063242] Key type lgssc registered [ 3523.150489] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3572.843402] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3572.891068] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3574.220809] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3574.308402] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3574.424769] Lustre: lustre-MDT0000: new disk, initializing [ 3574.670322] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3574.702833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3580.086366] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3592.675493] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3593.186244] Lustre: lustre-OST0000: new disk, initializing [ 3593.193825] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3593.199668] Lustre: Skipped 1 previous similar message [ 3593.210075] Lustre: 117991:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3593.314601] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3599.484146] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3599.494795] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3599.515402] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3600.385268] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3612.692665] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3612.858833] Lustre: lustre-OST0001: new disk, initializing [ 3612.863331] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3612.868920] Lustre: 118856:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3612.944046] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3619.488464] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3619.973482] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3619.985782] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3620.024412] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3630.055539] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3630.062086] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3630.070559] Lustre: Skipped 1 previous similar message [ 3630.086566] Lustre: Skipped 1 previous similar message [ 3634.660519] Lustre: server umount lustre-MDT0000 complete [ 3641.943840] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3647.233814] LustreError: 117176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787585398 with bad export cookie 4559911530816540 [ 3647.235463] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3647.250032] LustreError: 117176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3647.365809] Lustre: server umount lustre-OST0000 complete [ 3652.479032] Lustre: server umount lustre-OST0001 complete [ 3663.317102] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 11:30:11 (1787585411) [ 3688.564598] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 11:30:36 (1787585436) [ 3700.555813] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 3725.098797] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 11:31:13 (1787585473) [ 3770.203275] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 3780.303607] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 3824.677026] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3824.941530] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3824.999396] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3825.062435] Lustre: lustre-MDT0000: new disk, initializing [ 3825.128902] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3825.142684] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3830.269087] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3842.139063] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3842.282891] Lustre: 126611:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3842.367739] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3842.382228] Lustre: Skipped 1 previous similar message [ 3842.511486] Lustre: lustre-MDT0001: new disk, initializing [ 3842.624086] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3842.665824] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3842.681046] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3847.700953] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3857.950952] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3864.626664] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3872.613855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3872.938583] Lustre: lustre-OST0000: new disk, initializing [ 3872.946105] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3872.956464] Lustre: 128581:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3873.070645] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3881.034711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3881.046172] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3881.147406] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3881.185457] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3893.530486] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3898.855744] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3899.051729] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3905.209154] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3905.601363] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3912.161744] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3912.161905] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3912.181865] LustreError: Skipped 1 previous similar message [ 3912.186649] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3912.198862] Lustre: Skipped 1 previous similar message [ 3912.221231] Lustre: Skipped 1 previous similar message [ 3917.296544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3917.322688] Lustre: Skipped 1 previous similar message [ 3922.403754] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3922.406683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3922.432378] Lustre: Skipped 1 previous similar message [ 3922.615272] Lustre: server umount lustre-OST0000 complete [ 3927.523313] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3927.528516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3927.537352] Lustre: Skipped 1 previous similar message [ 3927.555373] Lustre: Skipped 1 previous similar message [ 3933.155591] Lustre: server umount lustre-MDT0000 complete [ 3937.296865] LustreError: 126605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787585688 with bad export cookie 4559911530817765 [ 3937.300471] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3937.307895] LustreError: 126605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3937.621282] Lustre: server umount lustre-MDT0001 complete [ 3946.993894] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 11:34:55 (1787585695) [ 3955.297571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3955.784684] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3960.743673] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3960.808822] LustreError: 130997:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3960.832639] LustreError: 130997:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3965.925787] LustreError: 130998:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3970.403237] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3975.983834] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3987.844722] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3997.572631] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4005.853378] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4006.431081] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4006.444702] Lustre: Skipped 1 previous similar message [ 4013.617115] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4022.062621] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4032.533500] Lustre: *** cfs_fail_loc=159, val=0*** [ 4032.539288] Lustre: Skipped 2 previous similar messages [ 4049.082555] Lustre: lustre-MDT0000: Client 6338d177-44c1-48e8-a844-fb95ef7cb34f (at 192.168.204.31@tcp) reconnecting [ 4051.363726] Lustre: *** cfs_fail_loc=159, val=0*** [ 4051.382600] Lustre: Skipped 3 previous similar messages [ 4067.580158] Lustre: lustre-MDT0000: Client 6338d177-44c1-48e8-a844-fb95ef7cb34f (at 192.168.204.31@tcp) reconnecting [ 4087.270235] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4087.282871] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4087.300142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4087.307303] Lustre: Skipped 2 previous similar messages [ 4090.723677] Lustre: server umount lustre-OST0000 complete [ 4098.016514] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4098.032223] Lustre: Skipped 2 previous similar messages [ 4107.234213] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4107.246182] Lustre: Skipped 6 previous similar messages [ 4110.303747] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4110.511814] Lustre: server umount lustre-MDT0000 complete [ 4112.356590] LustreError: 131727:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4112.384615] LustreError: 131727:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4115.605412] LustreError: 130978:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787585866 with bad export cookie 4559911530818647 [ 4115.606786] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4115.626214] LustreError: 130978:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4116.489502] Lustre: server umount lustre-MDT0001 complete [ 4128.661908] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 4132.596496] Key type lgssc unregistered [ 4132.907918] LNet: 134785:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4132.915855] LNetError: 134785:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4132.930845] LNet: Removed LNI 192.168.204.131@tcp [ 4134.523679] Key type .llcrypt unregistered [ 4134.540602] Key type ._llcrypt unregistered [ 4168.368934] Key type ._llcrypt registered [ 4168.372109] Key type .llcrypt registered [ 4168.522685] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 4169.677149] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4169.865241] alg: No test for adler32 (adler32-zlib) [ 4171.288695] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 4171.632599] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 4173.375315] Key type lgssc registered [ 4175.440184] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4186.663260] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4186.699032] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4188.484744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4196.030772] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4198.881057] LustreError: 136205:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4198.901660] LustreError: 136205:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4204.003073] LustreError: 136204:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4208.026134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4208.916808] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4215.817905] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4227.282073] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4235.495921] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4244.780340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4245.004031] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4252.959987] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4258.363104] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:97) [ 4263.148578] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4274.689126] Lustre: *** cfs_fail_loc=159, val=0*** [ 4291.283766] Lustre: lustre-MDT0000: Client 91f2270a-9be2-40ac-acf6-308e8d05e7a9 (at 192.168.204.31@tcp) reconnecting [ 4293.970810] Lustre: *** cfs_fail_loc=159, val=0*** [ 4310.248036] Lustre: lustre-MDT0001: Client 91f2270a-9be2-40ac-acf6-308e8d05e7a9 (at 192.168.204.31@tcp) reconnecting [ 4311.957072] Lustre: *** cfs_fail_loc=159, val=0*** [ 4311.960918] Lustre: Skipped 3 previous similar messages [ 4327.681340] Lustre: lustre-MDT0001: Client 91f2270a-9be2-40ac-acf6-308e8d05e7a9 (at 192.168.204.31@tcp) reconnecting [ 4333.442315] Lustre: 139247:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4337.349870] Lustre: *** cfs_fail_loc=159, val=0*** [ 4337.366732] Lustre: Skipped 4 previous similar messages [ 4353.769807] Lustre: lustre-MDT0000: Client 1d78c033-901d-4fe3-b8b0-fed6a5ae9884 (at 192.168.204.31@tcp) reconnecting [ 4355.930506] Lustre: *** cfs_fail_loc=159, val=0*** [ 4355.935314] Lustre: Skipped 14 previous similar messages [ 4371.659110] Lustre: lustre-MDT0000: Client 1d78c033-901d-4fe3-b8b0-fed6a5ae9884 (at 192.168.204.31@tcp) reconnecting [ 4373.467669] Lustre: 139536:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4380.134448] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4380.138818] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4380.166655] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4381.155682] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4381.164212] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4381.172761] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4381.207578] Lustre: Skipped 1 previous similar message [ 4385.966080] Lustre: server umount lustre-OST0000 complete [ 4391.392811] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4391.393631] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4391.411977] Lustre: Skipped 1 previous similar message [ 4392.469803] Lustre: server umount lustre-MDT0000 complete [ 4396.513331] LustreError: 136933:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4396.551218] LustreError: 136933:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4396.599564] LustreError: 136186:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787586147 with bad export cookie 1401466811116193323 [ 4396.614542] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4396.617568] LustreError: 136186:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4397.063396] Lustre: server umount lustre-MDT0001 complete [ 4409.798304] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 4412.644524] Key type lgssc unregistered [ 4413.000225] LNet: 140593:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4413.011582] LNetError: 140593:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4413.039051] LNet: Removed LNI 192.168.204.131@tcp [ 4414.093953] Key type .llcrypt unregistered [ 4414.098837] Key type ._llcrypt unregistered [ 4441.242158] Key type ._llcrypt registered [ 4441.248973] Key type .llcrypt registered [ 4441.361846] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 4442.468793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4442.680614] alg: No test for adler32 (adler32-zlib) [ 4443.801657] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 4444.086555] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 4445.799274] Key type lgssc registered [ 4446.868856] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4456.064599] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4456.090791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4457.711034] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4463.280083] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4468.193601] LustreError: 142010:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4468.219297] LustreError: 142010:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4473.345537] LustreError: 142010:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4474.358893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4475.102663] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4481.400715] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4492.454045] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4499.553823] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4507.549960] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4507.871993] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4514.927332] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4520.226902] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4524.046653] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4528.395145] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:117 to 0x280000401:161) [ 4535.375522] Lustre: 144597:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4539.936414] Lustre: 144645:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4541.898246] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4542.812857] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4546.876132] Lustre: 144793:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4547.825882] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4548.783698] Lustre: 144901:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4554.209461] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4554.235102] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4554.246438] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4554.247028] Lustre: Skipped 1 previous similar message [ 4559.329699] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4559.351226] Lustre: Skipped 1 previous similar message [ 4560.366561] Lustre: server umount lustre-OST0000 complete [ 4567.031192] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4567.039629] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4567.059224] Lustre: Skipped 1 previous similar message [ 4567.077699] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4569.570406] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4569.572178] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4569.593819] Lustre: Skipped 1 previous similar message [ 4571.803141] Lustre: server umount lustre-MDT0000 complete [ 4574.692576] LustreError: 142011:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4574.724352] LustreError: 142011:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4576.879129] LustreError: 144695:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787586327 with bad export cookie 10444370217205467356 [ 4576.890517] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4577.287390] Lustre: server umount lustre-MDT0001 complete [ 4595.926684] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 4599.671338] Key type lgssc unregistered [ 4600.139213] LNet: 145977:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4600.150424] LNetError: 145977:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4600.180218] LNet: Removed LNI 192.168.204.131@tcp [ 4601.334133] Key type .llcrypt unregistered [ 4601.335767] Key type ._llcrypt unregistered [ 4638.697848] Key type ._llcrypt registered [ 4638.703302] Key type .llcrypt registered [ 4638.806622] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 4639.937646] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4640.027855] alg: No test for adler32 (adler32-zlib) [ 4641.311755] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 4641.600283] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 4643.263676] Key type lgssc registered [ 4644.791503] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4655.688858] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4655.704289] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4657.406032] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4663.402445] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4667.878218] LustreError: 147396:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4667.907953] LustreError: 147396:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4672.993890] LustreError: 147397:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4674.321737] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4674.584469] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4680.637460] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4689.170825] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4696.445788] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4703.940442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4704.329438] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4711.973377] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4719.747703] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4721.808538] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4727.940050] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:117 to 0x280000401:193) [ 4732.733669] Lustre: *** cfs_fail_loc=159, val=0*** [ 4748.522608] Lustre: lustre-MDT0000: Client a5218405-fb20-4f1b-960d-f725846d210d (at 192.168.204.31@tcp) reconnecting [ 4769.248766] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4769.262692] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4769.268685] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4769.294863] Lustre: Skipped 1 previous similar message [ 4769.343664] Lustre: Skipped 1 previous similar message [ 4774.368552] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4774.390619] Lustre: Skipped 1 previous similar message [ 4779.492870] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4779.500183] Lustre: Skipped 1 previous similar message [ 4781.023782] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4781.469471] Lustre: server umount lustre-OST0000 complete [ 4787.173669] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4787.178931] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4787.203431] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4789.729340] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4793.626082] Lustre: server umount lustre-MDT0000 complete [ 4794.851356] LustreError: 148559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4794.884473] LustreError: 148559:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4798.240442] LustreError: 150321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787586549 with bad export cookie 4086784997020324903 [ 4798.251288] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4798.254037] LustreError: 150321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4798.998155] Lustre: server umount lustre-MDT0001 complete [ 4811.620788] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 4814.895995] Key type lgssc unregistered [ 4815.180597] LNet: 151076:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4815.189163] LNetError: 151076:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4815.215513] LNet: Removed LNI 192.168.204.131@tcp [ 4816.346844] Key type .llcrypt unregistered [ 4816.349258] Key type ._llcrypt unregistered [ 4848.017345] Key type ._llcrypt registered [ 4848.019671] Key type .llcrypt registered [ 4848.119360] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 4849.660099] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4849.798013] alg: No test for adler32 (adler32-zlib) [ 4850.891141] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 4851.210377] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 4852.943262] Key type lgssc registered [ 4854.282379] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4864.655731] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4864.699687] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4866.198312] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4872.371343] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4875.750801] LustreError: 152495:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4875.764075] LustreError: 152495:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4880.867548] LustreError: 152496:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4884.511607] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4884.893549] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4890.751203] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4900.782651] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4908.262566] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4917.780829] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4918.378650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4926.090972] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4929.645851] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4935.797867] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:202 to 0x280000401:225) [ 4936.648655] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4958.289855] Lustre: lustre-MDT0001: evicting 095a1417-d944-425e-b77c-3bbf3d9b8fee (at 192.168.204.31@tcp) by administrative request [ 4958.299140] Lustre: Skipped 1 previous similar message [ 4991.458199] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4991.470337] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4991.497097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4992.483911] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4992.485182] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4992.514535] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4994.565522] Lustre: server umount lustre-OST0000 complete [ 5002.726040] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5002.739863] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5002.751715] Lustre: Skipped 1 previous similar message [ 5002.752110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5002.752113] Lustre: Skipped 1 previous similar message [ 5006.082175] Lustre: server umount lustre-MDT0000 complete [ 5007.841082] LustreError: 154032:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5007.852336] LustreError: 154032:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5011.410409] LustreError: 152477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787586762 with bad export cookie 9001332767821010097 [ 5011.436317] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5011.775308] Lustre: server umount lustre-MDT0001 complete [ 5027.022478] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 5031.355260] Key type lgssc unregistered [ 5031.741758] LNet: 156576:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5031.753692] LNetError: 156576:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5031.790909] LNet: Removed LNI 192.168.204.131@tcp [ 5032.873603] Key type .llcrypt unregistered [ 5032.875542] Key type ._llcrypt unregistered [ 5125.103907] Key type ._llcrypt registered [ 5125.106779] Key type .llcrypt registered [ 5125.233639] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 5126.761088] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5126.793506] alg: No test for adler32 (adler32-zlib) [ 5127.945475] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 5128.257650] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 5129.983289] Key type lgssc registered [ 5131.329691] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5140.150737] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5140.200429] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5141.843464] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5146.275393] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5151.201213] LustreError: 159066:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5151.213033] LustreError: 159066:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5155.557415] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5155.889216] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5160.603631] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5169.909256] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5176.578646] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5183.629706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5184.059172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5187.144572] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:227 to 0x280000401:257) [ 5190.589877] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5198.077811] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5200.512250] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5212.640735] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5212.657465] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5212.685968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5216.225574] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5216.227568] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5216.245049] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5218.920319] Lustre: server umount lustre-OST0000 complete [ 5226.465374] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5226.489733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5226.504987] Lustre: Skipped 2 previous similar messages [ 5229.543136] Lustre: server umount lustre-MDT0000 complete [ 5231.589360] LustreError: 159066:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5231.612674] LustreError: 159066:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5233.914421] LustreError: 159048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787586984 with bad export cookie 4811338332355975767 [ 5233.919678] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5233.941787] LustreError: 159048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5234.289385] Lustre: server umount lustre-MDT0001 complete [ 5246.450836] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 5250.577374] Key type lgssc unregistered [ 5250.919502] LNet: 162632:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5250.936295] LNetError: 162632:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5250.974605] LNet: Removed LNI 192.168.204.131@tcp [ 5252.140959] Key type .llcrypt unregistered [ 5252.142981] Key type ._llcrypt unregistered [ 5279.587778] Key type ._llcrypt registered [ 5279.591574] Key type .llcrypt registered [ 5279.671987] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 5280.392484] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5280.479077] alg: No test for adler32 (adler32-zlib) [ 5281.657536] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 5281.980598] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 5283.633162] Key type lgssc registered [ 5284.872500] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5300.408559] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 11:57:28 (1787587048) [ 5317.514560] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 5328.514257] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 5371.664751] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5371.680819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5372.921851] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5372.963079] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5373.121544] Lustre: lustre-MDT0000: new disk, initializing [ 5373.277265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5373.307969] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5379.637173] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5393.223041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5393.364531] Lustre: 166899:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5393.463608] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5393.473647] Lustre: Skipped 1 previous similar message [ 5393.538735] Lustre: lustre-MDT0001: new disk, initializing [ 5393.625639] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5393.663057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5393.679462] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5399.213338] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5409.589378] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5415.738246] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5422.948158] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5423.325608] Lustre: lustre-OST0000: new disk, initializing [ 5423.331570] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5423.336582] Lustre: 168872:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5423.432423] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5430.106707] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5431.389438] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5431.405878] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5431.548867] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5438.782716] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5452.259816] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5452.273797] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5452.274056] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5452.290851] Lustre: Skipped 1 previous similar message [ 5452.313564] Lustre: Skipped 1 previous similar message [ 5457.384541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5457.395632] Lustre: Skipped 1 previous similar message [ 5457.933736] Lustre: server umount lustre-OST0000 complete [ 5462.502977] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5462.508589] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5462.520820] Lustre: Skipped 1 previous similar message [ 5462.534598] Lustre: Skipped 1 previous similar message [ 5467.623449] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5467.634543] Lustre: Skipped 1 previous similar message [ 5468.550208] Lustre: server umount lustre-MDT0000 complete [ 5471.907284] LustreError: 166892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787587222 with bad export cookie 2185633417295118724 [ 5471.908660] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5471.925219] LustreError: 166892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5472.203777] Lustre: server umount lustre-MDT0001 complete [ 5480.702459] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 5483.531551] Key type lgssc unregistered [ 5483.853724] LNet: 170779:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5483.864902] LNetError: 170779:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5483.899573] LNet: Removed LNI 192.168.204.131@tcp [ 5484.773298] Key type .llcrypt unregistered [ 5484.775597] Key type ._llcrypt unregistered [ 5511.218867] Key type ._llcrypt registered [ 5511.223103] Key type .llcrypt registered [ 5511.345599] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 5512.853829] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5512.991581] alg: No test for adler32 (adler32-zlib) [ 5514.247714] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 5514.496545] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 5516.151181] Key type lgssc registered [ 5517.101917] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5527.344504] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5527.374384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5528.877354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5533.157820] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5538.279507] LustreError: 172203:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5538.308941] LustreError: 172203:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5543.354896] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5543.395453] LustreError: 172204:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5543.757714] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5548.907402] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5556.828907] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5563.474317] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5571.128247] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5571.686399] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5573.753710] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5577.924092] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5586.126515] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5597.167526] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5597.198470] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5597.222368] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5597.241435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.272326] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5598.138755] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5598.158824] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5598.184974] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5598.194972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.210387] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5599.109463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.985886] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5599.992639] Lustre: Skipped 1 previous similar message [ 5600.010732] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5600.021654] Lustre: Skipped 1 previous similar message [ 5600.025272] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5600.031554] Lustre: Skipped 1 previous similar message [ 5600.043058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.056595] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5600.068845] Lustre: Skipped 1 previous similar message [ 5600.993823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.988852] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5601.993786] Lustre: Skipped 1 previous similar message [ 5602.009905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.046520] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5603.060160] Lustre: Skipped 2 previous similar messages [ 5603.064787] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5603.069822] Lustre: Skipped 2 previous similar messages [ 5603.074178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.098683] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5603.103939] Lustre: Skipped 2 previous similar messages [ 5604.294778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.434173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.546499] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5606.556393] Lustre: Skipped 3 previous similar messages [ 5606.587295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.857088] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5607.867226] Lustre: Skipped 3 previous similar messages [ 5607.877631] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5607.886203] Lustre: Skipped 3 previous similar messages [ 5607.889725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.897170] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5607.901490] Lustre: Skipped 3 previous similar messages [ 5608.760830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.804204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.763385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.065390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.301422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.652641] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5614.668952] Lustre: Skipped 6 previous similar messages [ 5614.704269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.083876] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5616.094210] Lustre: Skipped 6 previous similar messages [ 5616.098177] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5616.107243] Lustre: Skipped 6 previous similar messages [ 5616.110814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.125491] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5616.131462] Lustre: Skipped 6 previous similar messages [ 5617.217230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.109842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.190800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.113272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.238376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.279223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.166776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.179074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.220323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.220897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.513164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.392079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.216290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.267662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.364264] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5631.373365] Lustre: Skipped 15 previous similar messages [ 5631.388885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.365819] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5632.381681] Lustre: Skipped 15 previous similar messages [ 5632.385617] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5632.394989] Lustre: Skipped 15 previous similar messages [ 5632.418727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.459097] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5632.476034] Lustre: Skipped 15 previous similar messages [ 5633.395283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.550169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.582549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.860931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.073246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.381778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.857737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.196597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.779908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.569546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.893537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.259747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.233646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.599902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.709489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.650183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.922632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.038892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.983108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.250278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.093666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.056968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.066499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.192140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.157979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.197991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.099054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.150322] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5664.154543] Lustre: Skipped 28 previous similar messages [ 5664.169720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.161421] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5665.169995] Lustre: Skipped 28 previous similar messages [ 5665.173465] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5665.178308] Lustre: Skipped 28 previous similar messages [ 5665.182508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.196995] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5665.207119] Lustre: Skipped 28 previous similar messages [ 5666.403960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.483860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.423819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.601745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.676791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.779900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.722974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.716367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.653361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.495614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.391890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.342213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.165181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.095643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.106443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.172838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.164672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.230232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.134839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.152913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.976966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.904963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.841786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.684571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.545331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.552200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.460557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.576034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.404652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.392652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.348146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.312685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.379309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.352778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.486222] LustreError: 172223:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5699.508558] LustreError: 172223:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5699.534491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.715552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.759526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.638240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.580161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.679923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.923135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.746701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.595771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.631966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.762655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.943159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.064794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.073856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.089829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.065715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.183504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.167639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.215172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.220079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.329261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.443841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.601560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.624290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.592833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.488812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.384934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.306446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.506048] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5728.514882] Lustre: Skipped 63 previous similar messages [ 5728.541968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.615155] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5729.624963] Lustre: Skipped 63 previous similar messages [ 5729.630511] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5729.642107] Lustre: Skipped 63 previous similar messages [ 5729.649688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.658583] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5729.676942] Lustre: Skipped 63 previous similar messages [ 5730.594801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.519610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.640445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.556209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.443462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.418419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.455302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.240227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.069352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.039765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.074388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.126588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.105250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.168580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.150496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.292467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.149675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.262327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.376154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.303480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.353651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.164507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.997822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.942217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.889895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.933603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.975346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.941909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.928868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.802091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.759224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.139935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.172618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.307489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.559087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.454935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.022387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.090250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.249820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.171954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.098708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.099714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.962868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.950080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.884067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.787700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.594136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.500092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.485274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.460523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.294767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.205536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.327657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.212680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.131698] LustreError: 172223:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5784.154432] LustreError: 172223:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5784.157127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.258933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.179084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.252275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.212708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.234539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.233368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.898875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.043726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.933151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.250690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.703950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.979390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.171596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.641031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.746138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.124967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.224434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.555553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.628608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.735132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.745834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.737848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.868890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.115323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.193803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.298272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.603970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.667195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.755529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.942282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.125152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.171687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.007579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.411208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.626246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.472767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.759635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.726432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.961698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.041777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.044546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.155879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.124516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.189921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.156516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.276031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.076300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.855311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.842172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.017366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.925305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.094876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.318368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.395629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.494594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.848966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.956501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.954353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.364441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.665400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.099904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.240352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.520752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.844434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.816868] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5856.826230] Lustre: Skipped 120 previous similar messages [ 5856.850816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.452509] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5858.478588] Lustre: Skipped 120 previous similar messages [ 5858.485860] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5858.494146] Lustre: Skipped 120 previous similar messages [ 5858.522096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.555086] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5858.572310] Lustre: Skipped 120 previous similar messages [ 5859.860127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.104922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.367649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.396636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.285513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.302821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.352685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.346474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.286780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.558588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.446145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.523706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.523768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.404774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.098286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.207788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.331690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.400653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.509206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.581031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.725133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.846074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.895545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.943910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.935707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.054249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.294423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.352108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.518385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.652237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.732037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.488926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.478366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.278533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.091478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.209274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.269213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.296392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.440080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.310287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.286262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.242938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.260142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.201828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.267395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.290417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.299383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.326418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.235255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.324989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.492771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.638863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.877124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.033484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.091837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.097827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.019680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.140719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.512521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.032491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.362536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.342935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.294078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.354428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.344949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.378242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.950621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.144747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.127779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.441720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.804520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.008617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.292133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.286621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.080024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.989943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.978332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.944685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.999435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.026386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.901495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.776865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.065968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.863850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.899645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.838634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.911478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.024095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.846787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.663889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.530685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.500833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.341446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.224926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.085664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.087275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.954842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.799505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.729745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.696984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.446975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.239848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.209817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.162414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.165886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.983951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.812648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.793426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.824684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.781174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.664789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.455816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.396204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.377672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.251367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.234678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.041656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.114679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.966354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.861568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.783816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.947451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.258707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.376226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.294521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.572967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.569212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.781386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.922514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.949318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.312236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.469143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.514572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.028418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.276657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.039845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.121931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.929801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.791107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.546207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.672615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.692384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.985848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.006754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.225452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.617912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.659494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.544347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.472275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.644551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.586491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.601148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.621731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.492283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.463453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.352446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.442262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.573600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.687312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.557753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.662514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.566629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.701388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.788937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.817249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.026382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.287959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.601817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.635251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.582960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.507241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.578545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.585161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.612502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.582565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.587970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.842525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.870268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.662288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.412333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.372459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.577693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.628394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.558121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.261642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.173115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.907088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.795882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.676953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.808144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.602994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.429987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.176819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.871050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.546413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.279968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.025582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.830416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.758535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.748483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.168878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.512099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.439909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.609221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.610302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.626586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.730960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.675594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.397608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.410601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.456145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.506668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.506523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.675812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.753629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.994913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.119797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.170520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.164044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.992578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.794865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.775245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.873318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.784742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.844729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.768377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.938823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.918455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.806651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.768887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.102873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.167463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.349214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.457704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.482232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.777911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.687583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.971861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.856597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.863359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.040486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.352637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.688487] LustreError: 172223:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6107.700995] LustreError: 172223:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6107.711934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.971855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.114580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.521216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.666406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.757841] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6113.765109] Lustre: Skipped 248 previous similar messages [ 6113.788305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.976091] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6115.001306] Lustre: Skipped 248 previous similar messages [ 6115.015054] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6115.030150] Lustre: Skipped 248 previous similar messages [ 6115.043928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.065282] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6115.084111] Lustre: Skipped 248 previous similar messages [ 6116.246127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.280860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.362768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.324213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.330083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.658145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.762202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.857114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.281660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.453628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.542628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.648439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.783945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.119206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.441988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.699418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.791337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.850316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.035267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.160214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.186527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.545358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.610694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.630801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.561165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.508224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.563374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.574474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.480312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.433578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.475022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.406489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.266586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.208125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.018247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.974646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.920381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.912300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.890718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.928084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.744696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.822100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.689189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.975119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.005049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.898855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.916958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.892757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.207669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.278029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.509414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.793701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.946503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.972446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.902757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.856892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.743307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.608169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.526786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.445039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.291026] LustreError: 172223:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6179.307131] LustreError: 172223:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6179.320396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.407513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.324714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.278833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.459198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.481324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.709744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.902581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.805430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.764218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.701283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.875811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.934945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.923995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.039223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.038163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.114101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.086621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.134240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.084488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.002969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.162738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.196574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.362449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.401509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.362894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.449508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.715349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.766429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.890603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.889549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.034125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.109734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.157903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.184239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.053570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.025960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.931255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.992814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.035420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.104377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.916134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.079528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.201948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.222189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.238245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.230460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.250148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.154148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.994938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.108310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.003959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.938783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.016570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.827641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.724274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.619974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.804928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.739429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.608325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.682677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.715468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.582796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.769453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.862879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.897987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.002314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.987085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.909110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.887655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.932483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.944301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.902568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.855541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.061233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.324049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.357913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.401823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.293361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.326750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.188407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.503257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.855775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.177543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.488402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.697165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.778611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.678536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.844631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.849127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.040962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.116378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.294300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.304700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.565595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.849994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.038992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.109505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.117379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.074774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.120181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.070778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.277319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.656468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.809149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.939157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.092147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.253481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.550290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.445562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.460918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.344611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.275418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.343160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.455980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.822871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.895060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.800791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.741393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.602713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.558204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.519792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.873343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.253925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.533270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.587929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.941476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.995396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.167651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.341850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.108946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.109899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.173469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.228317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.336980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.625487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.838536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.872367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.758299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.912938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.014908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.173236] LustreError: 172223:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6329.190912] LustreError: 172223:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6329.201651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.715581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.978117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.064957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.392023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.669591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.760040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.634076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.709274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.034447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.235421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.287485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.478501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.424722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.581311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.732336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.594251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.734539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.848938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.861117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.915736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.767229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.824359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.854646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.833558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.002074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.066165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.880387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.770349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.746195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.742964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.760448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.848543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.830452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.755826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.810323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.784517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.717621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.598317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.587678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.551663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.443330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.193086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.093784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.938231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.953215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.820481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.646072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.515303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.356371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.194155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.007786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.889234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.820144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.697389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.590144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.500986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.339748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.310732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.116931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.992520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.868585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.750446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.594959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.453311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.411891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.260641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.170258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.121184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.978313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.786648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.577785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.554368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.508441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.449675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.265159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.322511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.328292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.318884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.315276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.332709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.228716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.527427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.617914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.898309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.195820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.395536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.364652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.446336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.513621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.632414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.025356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.994413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.246800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.511063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.611074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.711380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.622032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.742872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.817804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.851504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.881086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.862084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.165343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.430784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.664751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.684669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.700711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.693317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.890986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.210050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.299069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.463616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.599603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.620743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.607051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.769569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.819633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.791846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.936890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.924825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.133367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.258796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.321845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.283520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.452149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.328983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.373100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.491992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.631569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.764137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.978607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.323477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.557863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.880292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.334991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.616572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.947403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.168465] LustreError: 172223:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6473.184550] LustreError: 172223:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6473.224111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.563502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.596565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.926434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.153732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.101799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.420286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.919119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.271360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.632582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.871294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.071760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.273523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.309400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.410743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.452782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.865571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.896829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.075326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.194526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.392377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.755256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.838392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.884814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.959972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.998348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.864969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.843310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.073129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.069778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.100681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.075403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.210249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.130764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.070783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.037456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.171726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.430282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.393774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.356889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.236849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.316960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.254678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.340158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.560896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.637559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.603207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.530661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.667635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.493255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.361261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.463646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.489786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.557658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.861466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.848366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.248266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.455073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.502366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.563757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.782603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.055749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.296969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.468565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.554378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.716450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.802257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.078715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.229045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.495555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.698443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.883431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.959844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.974144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.291898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.703037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.884299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.896455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.883686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.970450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.093433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.223115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.258229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.334891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.628356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.744340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.932904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.119488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.130402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.239275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.426690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.575390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.091236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.468571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.858446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.306492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.192566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.207689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.226598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.347971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.443045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.790331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.096460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.357786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.285730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.260354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.212478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.585209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.593082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.733536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.871687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.905753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.224603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.346203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.414855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.645335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.689906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.750622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.616989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.819227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.839938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.000841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.196355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.364876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.652196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.740548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.895191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.878403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.237200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.513121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.585056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.590382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.766622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.629696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.677531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.776856] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6625.784093] Lustre: Skipped 475 previous similar messages [ 6625.828576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.855870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.803114] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6627.813709] Lustre: Skipped 476 previous similar messages [ 6627.817215] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6627.824111] Lustre: Skipped 476 previous similar messages [ 6627.828116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.841032] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6627.845909] Lustre: Skipped 476 previous similar messages [ 6628.870486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.052113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.360446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.534290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.836438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.940573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.351106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.516347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.684953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.862832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.209034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.489295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.019636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.064402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.292471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.730447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.102454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.488645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.499590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.069910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.971276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.083742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.802564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.058392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.495914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.715161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7153.754242] LustreError: 175479:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7153.777155] LustreError: 172223:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7898.599761] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7898.607959] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7898.619449] Lustre: Skipped 26 previous similar messages [ 7898.630312] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7898.637451] Lustre: Skipped 1 previous similar message [ 7903.721030] LustreError: 226269:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7903.751297] LustreError: 226269:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7903.944706] Lustre: server umount lustre-OST0000 complete [ 7908.834793] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7908.845034] Lustre: Skipped 1 previous similar message [ 7913.952805] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7913.963808] Lustre: Skipped 1 previous similar message [ 7914.984102] Lustre: server umount lustre-MDT0000 complete [ 7919.074922] LustreError: 172203:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7919.102651] LustreError: 172203:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7919.918743] LustreError: 172184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787589670 with bad export cookie 16851916510695198938 [ 7919.922851] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7919.933397] LustreError: 172184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7920.528534] Lustre: server umount lustre-MDT0001 complete [ 7930.824832] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 7933.635100] Key type lgssc unregistered [ 7933.909034] LNet: 227499:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7933.918969] LNetError: 227499:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7933.934821] LNet: Removed LNI 192.168.204.131@tcp [ 7934.756296] Key type .llcrypt unregistered [ 7934.758216] Key type ._llcrypt unregistered [ 7958.625771] Key type ._llcrypt registered [ 7958.629684] Key type .llcrypt registered [ 7958.718380] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 7959.853187] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7959.881711] alg: No test for adler32 (adler32-zlib) [ 7960.953308] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 7961.161532] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 7962.816774] Key type lgssc registered [ 7963.941632] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7972.663679] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7972.694565] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7974.235409] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7978.661760] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 7984.614954] LustreError: 228917:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7984.641067] LustreError: 228917:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7987.132524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7987.596409] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7991.849475] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 7999.731965] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8006.323848] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8014.551528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8015.065824] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8020.242424] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8022.427790] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8031.605532] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8036.812637] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8039.110490] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8039.126067] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8039.132942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8039.144564] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8039.145566] LustreError: 228936:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8042.976464] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8042.983793] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8043.002992] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8043.713328] Lustre: lustre-OST0000: Not available for connect from 192.168.204.31@tcp (stopping) [ 8046.049587] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8046.069697] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8047.441088] Lustre: server umount lustre-OST0000 complete [ 8059.208168] LustreError: 228936:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8081.380705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8081.754489] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8085.867655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8085.878830] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8088.652674] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8096.106663] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8096.125417] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 8096.317500] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8097.937452] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8103.666699] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8bdcc78db000.ost_server_uuid 50 [ 8105.362590] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8bdcc78db000.ost_server_uuid in FULL state after 0 sec [ 8110.560697] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8110.568985] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8110.581967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8110.586631] Lustre: Skipped 1 previous similar message [ 8111.596402] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8116.192743] Lustre: server umount lustre-OST0000 complete [ 8120.803163] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8120.812397] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8120.837675] Lustre: Skipped 1 previous similar message [ 8120.849275] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8120.857434] Lustre: Skipped 2 previous similar messages [ 8126.539755] Lustre: server umount lustre-MDT0000 complete [ 8126.946094] LustreError: 228916:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8126.970223] LustreError: 228916:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8130.614908] LustreError: 228897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787589881 with bad export cookie 1244524369272486681 [ 8130.616270] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8130.642289] LustreError: 228897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8131.036512] Lustre: server umount lustre-MDT0001 complete [ 8143.042854] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 8145.899627] Key type lgssc unregistered [ 8146.165686] LNet: 234127:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8146.172613] LNetError: 234127:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8146.190834] LNet: Removed LNI 192.168.204.131@tcp [ 8147.188909] Key type .llcrypt unregistered [ 8147.191114] Key type ._llcrypt unregistered [ 8183.691944] Key type ._llcrypt registered [ 8183.697716] Key type .llcrypt registered [ 8183.812980] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 8185.245555] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8185.437152] alg: No test for adler32 (adler32-zlib) [ 8186.586483] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 8186.880350] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 8188.640372] Key type lgssc registered [ 8189.852056] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8200.711457] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8200.733446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8202.126058] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8207.364701] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8211.426383] LustreError: 235856:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8211.447425] LustreError: 235856:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8216.553120] LustreError: 235857:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8217.414319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8217.693267] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8223.376779] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8227.077305] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8235.469852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8235.871885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8244.661578] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8247.156984] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8256.312061] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8256.580833] Lustre: lustre-OST0001: new disk, initializing [ 8256.584663] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8256.591927] Lustre: 238169:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8256.707187] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8263.393538] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8265.841983] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8265.854063] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8265.945260] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8274.964666] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8282.059717] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8299.489892] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8299.500538] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8299.512856] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8302.052373] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8302.054680] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8302.079725] Lustre: Skipped 1 previous similar message [ 8302.105553] Lustre: Skipped 2 previous similar messages [ 8303.594622] Lustre: server umount lustre-MDT0000 complete [ 8307.184665] LustreError: 235857:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8307.201323] LustreError: 235857:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8307.699058] LustreError: 239167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787590058 with bad export cookie 1137036272050289458 [ 8307.700859] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8307.728561] LustreError: 239167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8308.176387] Lustre: server umount lustre-MDT0001 complete [ 8323.178424] Lustre: server umount lustre-OST0000 complete [ 8338.362955] Lustre: server umount lustre-OST0001 complete [ 8365.726296] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 8378.733982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8379.357453] LustreError: 241278:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8379.477995] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8384.488944] LustreError: 241279:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8384.712620] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8394.720510] LustreError: 241279:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8394.765230] LustreError: 241279:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8397.325889] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8398.051878] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8404.535853] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8409.042877] Lustre: 242420:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8417.416051] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8417.891703] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8423.037125] LustreError: 242774:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8425.165184] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 8425.496285] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8435.265403] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8440.868715] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 8442.789468] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8451.782736] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8456.020674] Lustre: 244291:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8469.985078] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8469.992907] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8470.011019] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8470.018354] Lustre: Skipped 2 previous similar messages [ 8473.766582] Lustre: server umount mylustre-MDT0000 complete [ 8476.647603] LustreError: 241272:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8476.671445] LustreError: 241272:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 8477.892581] LustreError: 242799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787590228 with bad export cookie 1137036272050292265 [ 8477.901332] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8477.908357] LustreError: 242799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8478.300372] Lustre: server umount mylustre-MDT0001 complete [ 8493.229544] Lustre: server umount mylustre-OST0000 complete [ 8507.901196] Lustre: server umount mylustre-OST0001 complete [ 8538.398889] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 8553.069417] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8553.644974] LustreError: 246604:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8553.742435] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8553.757880] Lustre: Skipped 1 previous similar message [ 8558.999253] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8569.164435] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8574.816288] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8578.098460] Lustre: 247748:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8585.708979] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8593.373505] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8599.480213] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 8603.266807] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8608.769318] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 8611.764132] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8621.275516] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8625.745739] Lustre: 249622:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8636.385344] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8636.406112] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8636.429066] Lustre: Skipped 3 previous similar messages [ 8636.438613] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8636.446505] Lustre: Skipped 4 previous similar messages [ 8642.191444] Lustre: server umount tfs-MDT0000 complete [ 8647.191940] LustreError: 246580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787590398 with bad export cookie 1137036272050294155 [ 8647.195637] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8647.204911] LustreError: 246580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8647.426785] Lustre: server umount tfs-MDT0001 complete [ 8661.738864] Lustre: server umount tfs-OST0000 complete [ 8665.567347] Lustre: 235122:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787590400/real 1787590400] req@ffff9db781ee0a80 x1874423905994624/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787590416 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8665.605968] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8665.619157] Lustre: Skipped 3 previous similar messages [ 8666.909445] Lustre: server umount tfs-OST0001 complete [ 8694.258178] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 8705.038127] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8705.475374] LustreError: 251874:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8705.494245] LustreError: 251874:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 8705.585730] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8705.590687] Lustre: Skipped 3 previous similar messages [ 8711.084405] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8722.766846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8729.801745] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8733.540935] Lustre: 253017:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8741.409831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8749.922994] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8751.114142] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 8760.300241] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8765.966546] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 8767.769560] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8776.647801] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8780.695401] Lustre: 254890:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8788.142848] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 12:55:36 (1787590536) [ 8805.347355] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8805.353571] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8805.376581] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8805.382309] Lustre: Skipped 4 previous similar messages [ 8810.969582] Lustre: server umount lustre-MDT0000 complete [ 8815.110025] LustreError: 251851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787590565 with bad export cookie 1137036272050295821 [ 8815.131744] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8815.137735] LustreError: 251851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8817.129711] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8817.141338] Lustre: Skipped 5 previous similar messages [ 8821.569274] Lustre: server umount lustre-MDT0001 complete [ 8832.297306] Lustre: server umount lustre-OST0000 complete [ 8843.085830] Lustre: server umount lustre-OST0001 complete [ 8851.612066] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 8863.482892] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 8907.439887] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8907.754996] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8907.785242] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8907.879282] Lustre: lustre-MDT0000: new disk, initializing [ 8907.967927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8907.973015] Lustre: Skipped 3 previous similar messages [ 8907.999515] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8913.547106] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8927.444932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8927.697157] Lustre: 258078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8927.856421] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8927.860173] Lustre: Skipped 1 previous similar message [ 8927.952509] Lustre: lustre-MDT0001: new disk, initializing [ 8928.169282] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8928.197412] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8933.892840] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8947.620833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8948.020258] Lustre: lustre-OST0000: new disk, initializing [ 8948.031060] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8948.037235] Lustre: 259663:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8949.104525] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8949.123810] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8949.289538] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8957.113901] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8971.202812] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8971.427277] Lustre: lustre-OST0001: new disk, initializing [ 8971.430854] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8971.444377] Lustre: 260538:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8973.503052] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8973.516417] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8973.621850] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8978.998028] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8998.114319] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 12:59:05 (1787590745) [ 9018.340909] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 12:59:26 (1787590766) [ 9035.241311] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9035.255835] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9035.262628] Lustre: Skipped 6 previous similar messages [ 9035.269337] Lustre: Skipped 2 previous similar messages [ 9040.837604] Lustre: server umount lustre-OST0000 complete [ 9045.474261] LustreError: 259644:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9045.508315] LustreError: 259644:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 16 previous similar messages [ 9051.648756] Lustre: server umount lustre-MDT0000 complete [ 9055.837255] LustreError: 258873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787590806 with bad export cookie 1137036272050297375 [ 9055.852890] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9055.859842] LustreError: 258873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9062.502817] Lustre: server umount lustre-MDT0001 complete [ 9075.918863] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 9155.832378] Lustre: server umount lustre-OST0001 complete [ 9163.829137] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 9181.822305] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 9225.797782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9226.063606] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9226.118579] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9226.283981] Lustre: lustre-MDT0000: new disk, initializing [ 9226.417972] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9226.424113] Lustre: Skipped 3 previous similar messages [ 9226.445528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9231.883716] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9244.074895] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9244.216967] Lustre: 265430:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9244.283829] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9244.288759] Lustre: Skipped 1 previous similar message [ 9244.372683] Lustre: lustre-MDT0001: new disk, initializing [ 9244.506990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9244.536386] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9249.601488] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9261.006812] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9268.053782] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9275.649532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9275.902162] Lustre: lustre-OST0000: new disk, initializing [ 9275.905588] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9275.912350] Lustre: 267401:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9277.597893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9277.611358] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9277.690254] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9282.633119] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9293.635703] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9303.530508] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9303.533570] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9303.549858] Lustre: Skipped 5 previous similar messages [ 9303.550180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9303.550184] Lustre: Skipped 10 previous similar messages [ 9308.195930] Lustre: server umount lustre-OST0000 complete [ 9318.373481] Lustre: server umount lustre-MDT0000 complete [ 9322.926925] LustreError: 265424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787591073 with bad export cookie 1137036272050298845 [ 9322.928972] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9322.934597] LustreError: 265424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9323.359905] Lustre: server umount lustre-MDT0001 complete [ 9336.547500] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 9339.615304] Key type lgssc unregistered [ 9339.971628] LNet: 269164:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9339.978366] LNetError: 269164:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9340.014335] LNet: Removed LNI 192.168.204.131@tcp [ 9341.041253] Key type .llcrypt unregistered [ 9341.043980] Key type ._llcrypt unregistered [ 9369.094812] Key type ._llcrypt registered [ 9369.098622] Key type .llcrypt registered [ 9369.215102] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 9370.387422] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9370.429054] alg: No test for adler32 (adler32-zlib) [ 9371.690912] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 9372.029383] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 9373.759757] Key type lgssc registered [ 9375.369505] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9384.827702] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9384.856138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9386.578358] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9391.241481] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9396.720330] LustreError: 270628:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9396.736943] LustreError: 270628:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9399.845786] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9400.230871] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9404.700329] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9412.313543] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9419.117590] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9427.576560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9427.893241] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9431.992957] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9436.678704] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9447.548418] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9448.942682] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9452.513714] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9452.522817] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9452.538622] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9454.564593] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9454.567329] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9454.609145] Lustre: Skipped 1 previous similar message [ 9457.749123] Lustre: server umount lustre-OST0000 complete [ 9464.806533] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9464.813127] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9464.827233] Lustre: Skipped 1 previous similar message [ 9464.856354] Lustre: Skipped 1 previous similar message [ 9468.727311] Lustre: server umount lustre-MDT0000 complete [ 9469.934354] LustreError: 270624:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9469.969492] LustreError: 270624:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9473.123818] LustreError: 272471:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787591223 with bad export cookie 5521865547923911239 [ 9473.135710] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9473.137742] LustreError: 272471:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9473.728737] Lustre: server umount lustre-MDT0001 complete [ 9487.773673] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 9491.323854] Key type lgssc unregistered [ 9491.886765] LNet: 274217:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9491.897077] LNetError: 274217:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9491.948955] LNet: Removed LNI 192.168.204.131@tcp [ 9493.188375] Key type .llcrypt unregistered [ 9493.194458] Key type ._llcrypt unregistered [ 9514.011327] Key type ._llcrypt registered [ 9514.013779] Key type .llcrypt registered [ 9514.169190] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 9515.606084] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9515.657260] alg: No test for adler32 (adler32-zlib) [ 9516.795334] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 9517.077245] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 9518.823182] Key type lgssc registered [ 9520.365778] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9531.284098] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9531.316763] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9547.104754] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9552.223375] LustreError: 275412:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.131@tcp: failed processing log, type 4: rc = -110 [ 9579.039278] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9586.736291] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9596.216719] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9599.413880] Lustre: server umount lustre-OST0000 complete [ 9619.028934] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 9622.256608] Key type lgssc unregistered [ 9622.595683] LNet: 277046:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9622.607317] LNetError: 277046:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9622.629799] LNet: Removed LNI 192.168.204.131@tcp [ 9623.640782] Key type .llcrypt unregistered [ 9623.642646] Key type ._llcrypt unregistered [ 9671.217789] Key type ._llcrypt registered [ 9671.220974] Key type .llcrypt registered [ 9671.395709] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 9672.728319] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9672.869995] alg: No test for adler32 (adler32-zlib) [ 9673.945666] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [ 9674.182408] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 9675.855220] Key type lgssc registered [ 9677.186738] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9692.449227] loop0: detected capacity change from 0 to 204800000 [ 9692.680817] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9692.696318] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9693.071201] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9694.290717] loop0: detected capacity change from 0 to 204800000 [ 9694.445830] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9694.453840] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9694.731766] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9695.866752] loop0: detected capacity change from 0 to 204800000 [ 9696.069072] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9696.082836] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9696.270137] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9697.455675] loop0: detected capacity change from 0 to 204800000 [ 9697.640652] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9697.772790] print_req_error: 1 callbacks suppressed [ 9697.772800] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9698.959526] loop0: detected capacity change from 0 to 204800000 [ 9698.998024] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9703.376595] loop0: detected capacity change from 0 to 204800000 [ 9703.411025] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9708.406875] loop0: detected capacity change from 0 to 204800000 [ 9708.447343] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9713.226283] loop0: detected capacity change from 0 to 204800000 [ 9713.246324] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9717.739932] loop0: detected capacity change from 0 to 204800000 [ 9717.808684] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9717.865246] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9718.039962] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9718.072711] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9718.096926] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9718.117040] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9718.132840] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9718.146768] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9718.157595] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9718.184113] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9718.196327] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9718.213390] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9718.221847] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9718.232927] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9718.252266] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9718.283115] Lustre: 280466:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9724.849762] Lustre: server umount MGS complete [ 9725.890652] loop0: detected capacity change from 0 to 204800000 [ 9725.980481] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9726.590191] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9726.661299] Lustre: lustre-MDT0000: Aborting client recovery [ 9727.589730] loop1: detected capacity change from 0 to 204800000 [ 9727.764385] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9727.904617] Lustre: 280897:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9727.912521] Lustre: 280897:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9727.920083] Lustre: 280897:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9727.931341] Lustre: 280897:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9727.941359] Lustre: 280897:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9727.951585] Lustre: 280897:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9727.961508] Lustre: 280897:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9727.983893] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9728.180905] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9728.223245] Lustre: lustre-MDT0001: Aborting client recovery [ 9729.477501] loop2: detected capacity change from 0 to 204800000 [ 9729.657593] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9729.796251] Lustre: 280975:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9729.811417] Lustre: 280975:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9729.838945] Lustre: 280975:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9729.856243] Lustre: 280975:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9729.871761] Lustre: 280975:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9729.886708] Lustre: 280975:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9729.904485] Lustre: 280975:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9730.099073] Lustre: Found index 0 for lustre-OST0000, updating log [ 9730.154664] Lustre: 280994:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9730.248460] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9730.266211] Lustre: lustre-OST0000: Aborting recovery [ 9731.305407] loop3: detected capacity change from 0 to 204800000 [ 9731.453677] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9731.581248] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9731.598731] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9731.619786] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9731.646627] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9731.668069] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9731.686754] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9731.701201] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9731.731396] Lustre: Found index 1 for lustre-OST0001, updating log [ 9731.767601] Lustre: 281065:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9731.884683] Lustre: lustre-OST0001: Aborting recovery [ 9736.999745] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9737.013219] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9737.030136] LustreError: 280841:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9737.073408] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9737.120867] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9738.025538] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9738.026695] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9741.331863] LustreError: 280808:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 917us: rc = -2 [ 9741.879972] LustreError: 280809:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 424us: rc = -2 [ 9741.906277] LustreError: 280809:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 9758.694331] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9758.700198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9758.716591] Lustre: Skipped 2 previous similar messages [ 9758.720119] Lustre: Skipped 1 previous similar message [ 9761.296975] Lustre: server umount lustre-MDT0000 complete [ 9762.387071] LustreError: 280792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787591513 with bad export cookie 236691024933658337 [ 9762.412037] LustreError: 280792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9762.413402] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9763.810147] LustreError: 280805:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9763.811917] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9763.822084] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9763.822098] Lustre: Skipped 2 previous similar messages [ 9763.833399] LustreError: 280805:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9763.883278] Lustre: Skipped 2 previous similar messages [ 9767.840475] LustreError: 280806:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9767.840602] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9767.869678] LustreError: 280806:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9767.889424] Lustre: Skipped 2 previous similar messages [ 9768.960557] Lustre: server umount lustre-MDT0001 complete [ 9784.287207] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9784.466767] Lustre: server umount lustre-OST0000 complete [ 9791.743637] Lustre: server umount lustre-OST0001 complete [ 9800.188802] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 13:12:28 (1787591548) [ 9831.496572] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 9844.125833] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 9886.143132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9886.434961] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9886.478884] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9886.614937] Lustre: lustre-MDT0000: new disk, initializing [ 9886.699626] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9886.704706] Lustre: Skipped 1 previous similar message [ 9886.720673] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9892.454964] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9904.611067] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9904.806330] Lustre: 285134:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9904.894468] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9904.900351] Lustre: Skipped 1 previous similar message [ 9905.017187] Lustre: lustre-MDT0001: new disk, initializing [ 9905.148152] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9905.229139] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9905.251926] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9910.546806] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9922.010567] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9929.798558] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9938.593130] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9939.032193] Lustre: lustre-OST0000: new disk, initializing [ 9939.045220] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9939.058516] Lustre: 287106:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9939.162748] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9945.178192] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9945.188433] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9945.314261] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9946.290424] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9957.941762] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9964.814969] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9975.553788] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9980.007681] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9985.997888] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10037.729645] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10037.733246] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10037.735071] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10037.750757] Lustre: Skipped 1 previous similar message [10039.240785] Lustre: server umount lustre-OST0000 complete [10043.361342] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10043.394786] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10043.426044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10043.434515] Lustre: Skipped 1 previous similar message [10047.971788] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10049.767651] Lustre: server umount lustre-MDT0000 complete [10053.102873] LustreError: 286279:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10053.143686] LustreError: 286279:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10054.920297] LustreError: 285127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787591805 with bad export cookie 236691024933661053 [10054.927414] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10054.930396] LustreError: 285127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [10055.534541] Lustre: server umount lustre-MDT0001 complete [10066.662331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10072.022456] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10078.195333] Lustre: server umount MGS complete [10090.996201] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10091.476120] LustreError: 290875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10091.516840] LustreError: 290888:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10091.659047] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10096.702758] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10097.127648] LustreError: 290876:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10106.342628] LustreError: 290875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10106.360779] LustreError: 290875:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10106.877558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10113.139959] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10122.611252] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10129.650714] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10137.911274] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10138.445741] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10138.461594] Lustre: Skipped 1 previous similar message [10147.901725] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10158.377157] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10197.995079] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10198.000459] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10198.022595] Lustre: Skipped 1 previous similar message [10198.030088] Lustre: Skipped 2 previous similar messages [10200.903069] Lustre: server umount lustre-OST0000 complete [10211.201458] Lustre: server umount lustre-MDT0000 complete [10213.360315] LustreError: 290875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10213.393451] LustreError: 290875:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10215.595500] LustreError: 290858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787591966 with bad export cookie 236691024933662894 [10215.619743] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10215.925385] Lustre: server umount lustre-MDT0001 complete [10229.062454] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [10232.294896] Key type lgssc unregistered [10232.633094] LNet: 294825:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10232.639437] LNetError: 294825:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10232.667331] LNet: Removed LNI 192.168.204.131@tcp [10233.930055] Key type .llcrypt unregistered [10233.932493] Key type ._llcrypt unregistered [10280.016979] Key type ._llcrypt registered [10280.020967] Key type .llcrypt registered [10280.186426] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [10300.231334] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [10301.070116] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10301.187895] alg: No test for adler32 (adler32-zlib) [10302.249929] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [10302.603830] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [10304.359184] Key type lgssc registered [10305.432767] Lustre: Echo OBD driver; http://www.lustre.org/ [10355.308227] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10355.335734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10356.683693] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10356.763744] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10356.830863] Lustre: lustre-MDT0000: new disk, initializing [10356.910839] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10356.925991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10363.534954] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10376.050612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10376.216713] Lustre: 298790:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10376.291799] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10376.297926] Lustre: Skipped 1 previous similar message [10376.401441] Lustre: lustre-MDT0001: new disk, initializing [10376.525888] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10376.584638] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10376.595551] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10382.646268] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10395.981239] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10403.658401] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10412.581789] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10412.935271] Lustre: lustre-OST0000: new disk, initializing [10412.947727] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10412.955547] Lustre: 300759:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10413.081879] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10420.565545] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10422.433841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10422.456765] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10422.617309] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10431.622094] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10440.263709] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10451.849169] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10456.595978] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10462.549644] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10509.825221] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10509.833759] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10509.855355] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10509.864510] Lustre: Skipped 1 previous similar message [10512.510818] Lustre: server umount lustre-OST0000 complete [10520.032195] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10520.040341] LustreError: Skipped 1 previous similar message [10520.040828] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10520.046362] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10520.063135] Lustre: Skipped 2 previous similar messages [10525.156600] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10525.168143] Lustre: Skipped 2 previous similar messages [10529.249391] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10529.256304] Lustre: Skipped 1 previous similar message [10531.296603] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10531.596329] Lustre: server umount lustre-MDT0000 complete [10535.325566] LustreError: 301962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787592286 with bad export cookie 503746670229811970 [10535.330747] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10535.335069] LustreError: 301962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10535.737534] Lustre: server umount lustre-MDT0001 complete [10545.166896] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10549.775800] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10555.463537] Lustre: server umount MGS complete [10567.366136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10567.872113] LustreError: 304559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10567.899447] LustreError: 304559:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10567.917161] LustreError: 304572:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10567.971987] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10572.600098] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10573.283849] LustreError: 304560:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10577.395356] LustreError: 304559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10581.792051] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10582.335924] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10588.055735] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10597.167819] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10603.275271] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10610.133750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10610.521942] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10617.461234] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10625.844289] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10657.766020] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10657.779587] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10657.785302] Lustre: Skipped 1 previous similar message [10657.805427] Lustre: Skipped 1 previous similar message [10662.889423] LustreError: 306382:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10662.923096] LustreError: 306382:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10663.160253] Lustre: server umount lustre-OST0000 complete [10668.001679] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10668.024197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10668.034054] Lustre: Skipped 1 previous similar message [10682.335148] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10682.620374] Lustre: server umount lustre-MDT0000 complete [10683.362420] LustreError: 304560:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10683.390045] LustreError: 304560:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10686.656408] LustreError: 307285:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787592437 with bad export cookie 503746670229813909 [10686.661968] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10686.675933] LustreError: 307285:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10687.249043] Lustre: server umount lustre-MDT0001 complete [10700.037782] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [10703.483632] Key type lgssc unregistered [10703.850809] LNet: 308510:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10703.854598] LNetError: 308510:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10703.868445] LNet: Removed LNI 192.168.204.131@tcp [10704.556546] Key type .llcrypt unregistered [10704.559919] Key type ._llcrypt unregistered [10729.752674] Key type ._llcrypt registered [10729.760549] Key type .llcrypt registered [10729.892809] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [10745.942716] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [10747.783275] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10747.838095] alg: No test for adler32 (adler32-zlib) [10749.029150] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [10749.378050] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [10751.055207] Key type lgssc registered [10752.269340] Lustre: Echo OBD driver; http://www.lustre.org/ [10777.236703] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [10791.233710] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10791.294798] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10792.630721] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10792.709489] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10792.786688] Lustre: lustre-MDT0000: new disk, initializing [10792.841131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10792.856860] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10797.232467] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10808.971801] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10809.325621] Lustre: lustre-OST0000: new disk, initializing [10809.334633] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10809.341444] Lustre: Skipped 1 previous similar message [10809.348925] Lustre: 312562:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10809.410440] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10811.845292] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10811.858256] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10811.899985] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10816.170119] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11829.349251] Lustre: server umount lustre-MDT0000 complete [11833.952924] LustreError: 311750:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787593584 with bad export cookie 11761724168720213702 [11833.961756] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11843.791573] Lustre: server umount lustre-OST0000 complete [11868.581568] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [11879.174106] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [11920.610706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11920.788773] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11920.842944] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11920.938670] Lustre: lustre-MDT0000: new disk, initializing [11921.020055] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11921.040056] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11925.478960] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11936.681513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11936.816720] Lustre: 316923:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11936.867188] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11936.870971] Lustre: Skipped 1 previous similar message [11936.932917] Lustre: lustre-MDT0001: new disk, initializing [11937.035964] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11937.077637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11937.100489] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11941.930924] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11952.515199] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11959.229365] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11967.154043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11967.524390] Lustre: lustre-OST0000: new disk, initializing [11967.531063] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11967.537190] Lustre: 318891:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11967.625900] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11974.196736] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11977.267459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11977.279936] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11977.409073] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11983.629806] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11988.361542] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11988.600221] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11992.559883] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11992.776931] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11998.176619] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11998.181054] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11998.188868] LustreError: Skipped 1 previous similar message [11998.196394] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11998.214760] Lustre: Skipped 1 previous similar message [12000.744625] Lustre: server umount lustre-OST0000 complete [12008.424324] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12008.431709] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12008.446727] Lustre: Skipped 1 previous similar message [12008.454656] Lustre: Skipped 1 previous similar message [12011.394277] Lustre: server umount lustre-MDT0000 complete [12015.445405] LustreError: 316916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787593766 with bad export cookie 11761724168723154514 [12015.449141] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12015.458241] LustreError: 316916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12015.831487] Lustre: server umount lustre-MDT0001 complete [12019.162968] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [12021.274328] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 13:49:29 (1787593769) [12028.156815] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12028.550737] LustreError: 321204:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12028.564910] LustreError: 321204:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12028.632192] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12033.507820] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12034.018971] LustreError: 321205:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12039.147145] LustreError: 321204:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12042.073632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12042.434358] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12046.897054] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12055.832162] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12062.298582] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12069.809605] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12070.116830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12077.274547] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12086.612322] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12095.309677] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12095.521519] Lustre: lustre-OST0001: new disk, initializing [12095.535905] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12095.548984] Lustre: 324041:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12095.691728] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12104.050277] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12105.382760] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12105.397253] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12105.466300] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12105.483744] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12106.532419] LustreError: 324460:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12106.545147] LustreError: 324460:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12107.577041] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12109.666406] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12109.699666] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [12113.825099] LustreError: 324460:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12113.840739] LustreError: 324460:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [12115.769542] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12115.999389] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12122.028396] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12122.043945] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [12122.165342] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8bdcc7894000.ost_server_uuid 50 [12123.737219] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8bdcc7894000.ost_server_uuid in FULL state after 0 sec [12129.905594] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8bdcc7894000.ost_server_uuid 50 [12131.379698] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8bdcc7894000.ost_server_uuid in FULL state after 0 sec [12138.487072] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12148.704182] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12148.716120] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12148.743062] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12148.749396] Lustre: Skipped 1 previous similar message [12151.483177] Lustre: lustre-OST0001: Not available for connect from 192.168.204.31@tcp (stopping) [12151.491700] Lustre: Skipped 1 previous similar message [12152.833847] Lustre: server umount lustre-OST0001 complete [12156.573350] LustreError: 323024:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.31@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12159.464283] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12159.482038] LustreError: Skipped 1 previous similar message [12159.486153] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12159.501472] Lustre: Skipped 1 previous similar message [12159.514875] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12159.526992] Lustre: Skipped 2 previous similar messages [12162.023851] LustreError: 323027:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12162.046886] LustreError: 323027:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12165.376986] Lustre: server umount lustre-OST0000 complete [12170.726151] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12170.733515] LustreError: Skipped 1 previous similar message [12170.736584] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12170.747487] Lustre: Skipped 1 previous similar message [12170.751943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12170.757264] Lustre: Skipped 3 previous similar messages [12176.511426] Lustre: server umount lustre-MDT0000 complete [12177.382300] LustreError: 321204:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12177.416299] LustreError: 321204:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12181.957953] LustreError: 321185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787593932 with bad export cookie 11761724168723155403 [12181.963316] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12181.981389] LustreError: 321185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12182.528679] Lustre: server umount lustre-MDT0001 complete [12195.409215] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [12198.282291] Key type lgssc unregistered [12198.553576] LNet: 326736:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12198.559743] LNetError: 326736:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12198.570511] LNet: Removed LNI 192.168.204.131@tcp [12199.442559] Key type .llcrypt unregistered [12199.445387] Key type ._llcrypt unregistered [12230.398335] Key type ._llcrypt registered [12230.406717] Key type .llcrypt registered [12230.542232] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [12232.952655] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12232.992404] alg: No test for adler32 (adler32-zlib) [12234.047467] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [12234.248364] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [12235.967284] Key type lgssc registered [12237.754181] Lustre: Echo OBD driver; http://www.lustre.org/ [12247.073898] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12247.097708] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12248.500834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12252.821912] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12258.787072] LustreError: 328158:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12258.804114] LustreError: 328158:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12260.904597] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12263.908786] LustreError: 328159:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12267.482847] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12267.838121] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12271.927247] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12281.475368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12281.731379] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12287.230504] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12292.973882] LustreError: 329838:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12293.999113] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12296.491889] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12299.239965] LustreError: 329839:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12299.264536] LustreError: 329839:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12304.360390] LustreError: 330423:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12304.576610] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12304.893687] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12310.016578] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12311.666586] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12319.993927] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12327.751643] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12327.946664] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12437.988449] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12438.022767] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12438.024976] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12438.050348] Lustre: Skipped 1 previous similar message [12441.063885] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12441.085642] Lustre: Skipped 1 previous similar message [12446.182672] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12446.191144] Lustre: Skipped 1 previous similar message [12450.271174] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12450.342138] Lustre: server umount lustre-OST0001 complete [12451.299348] LustreError: 329838:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12451.316557] LustreError: 329838:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12457.952434] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12457.960219] LustreError: Skipped 1 previous similar message [12457.962174] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12457.977157] Lustre: Skipped 1 previous similar message [12457.985200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12461.541185] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12466.666742] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12466.684678] Lustre: Skipped 3 previous similar messages [12471.263140] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12471.565445] Lustre: server umount lustre-OST0000 complete [12476.899909] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12476.910077] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12476.922479] Lustre: Skipped 1 previous similar message [12476.943470] Lustre: Skipped 1 previous similar message [12481.917478] Lustre: server umount lustre-MDT0000 complete [12482.020298] LustreError: 328154:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12482.052520] LustreError: 328154:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [12486.121319] LustreError: 328139:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787594236 with bad export cookie 3658060843951695203 [12486.122752] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12486.138332] LustreError: 328139:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12486.541191] Lustre: server umount lustre-MDT0001 complete [12501.755855] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [12506.562503] Key type lgssc unregistered [12507.067903] LNet: 333112:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12507.085933] LNetError: 333112:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12507.156756] LNet: Removed LNI 192.168.204.131@tcp [12508.700917] Key type .llcrypt unregistered [12508.709633] Key type ._llcrypt unregistered [12566.881572] Key type ._llcrypt registered [12566.884580] Key type .llcrypt registered [12566.968506] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [12567.864359] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12567.874754] alg: No test for adler32 (adler32-zlib) [12568.929665] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [12569.155660] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [12570.839799] Key type lgssc registered [12571.940269] Lustre: Echo OBD driver; http://www.lustre.org/ [12580.413777] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12580.429424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12582.060806] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12586.305928] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12591.588875] LustreError: 335404:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12591.606939] LustreError: 335404:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12595.463395] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12595.868689] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12600.021813] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12603.061903] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12611.110966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12611.598109] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12619.126562] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12620.853403] LustreError: 336892:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12625.896390] LustreError: 336894:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12626.958647] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12629.486294] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12629.798497] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12635.143839] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12637.078881] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12646.815590] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12653.205497] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12665.828604] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12665.841660] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12665.859531] Lustre: Skipped 2 previous similar messages [12665.874547] Lustre: Skipped 3 previous similar messages [12670.944103] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12670.951409] Lustre: Skipped 1 previous similar message [12672.082144] Lustre: server umount lustre-MDT0000 complete [12675.393931] LustreError: 335386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787594426 with bad export cookie 17974681443828553605 [12675.394422] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12675.405766] LustreError: 335386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12675.771825] Lustre: server umount lustre-MDT0001 complete [12689.836166] Lustre: server umount lustre-OST0000 complete [12691.935472] Lustre: 334668:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787594426/real 1787594426] req@ffff9db79015f100 x1874428500704896/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787594442 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12691.958532] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12691.972439] Lustre: Skipped 1 previous similar message [12693.492646] Lustre: server umount lustre-OST0001 complete [12709.991685] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [12720.238250] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12720.633824] LustreError: 340682:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12720.650268] LustreError: 340682:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12720.716449] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12725.094618] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12725.729924] LustreError: 340683:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12733.533656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12733.915831] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12738.727467] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12741.593686] Lustre: 341822:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12748.687051] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12756.031770] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12758.453704] LustreError: 342177:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12758.481732] LustreError: 342177:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12765.693989] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12767.051331] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12767.347494] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12767.360438] Lustre: Skipped 1 previous similar message [12772.882184] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12773.864727] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12781.944474] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12786.183910] Lustre: 343695:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12793.341066] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12793.366621] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12793.369254] Lustre: Skipped 1 previous similar message [12793.392717] Lustre: Skipped 5 previous similar messages [12797.278075] Lustre: server umount lustre-MDT0000 complete [12798.434590] LustreError: 340679:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12798.461114] LustreError: 340679:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [12801.512905] LustreError: 340663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787594552 with bad export cookie 17974681443828555306 [12801.516923] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12801.536060] LustreError: 340663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12802.036801] Lustre: server umount lustre-MDT0001 complete [12817.200891] Lustre: server umount lustre-OST0000 complete [12819.743378] Lustre: 334669:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787594554/real 1787594554] req@ffff9db78d93a680 x1874428500755712/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787594570 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12819.771287] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12819.784421] Lustre: Skipped 2 previous similar messages [12821.894158] Lustre: server umount lustre-OST0001 complete [12825.388691] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12826.990374] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12828.929972] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 14:02:57 (1787594577) [12853.772374] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [12863.849639] loop0: detected capacity change from 0 to 1099511627776 [12864.618275] SGI XFS with ACLs, security attributes, quota, no debug enabled [12864.624297] *** VALIDATE xfs *** [12864.631773] XFS (loop0): Mounting V5 Filesystem [12864.657302] XFS (loop0): Ending clean mount [12865.950253] loop1: detected capacity change from 0 to 18691697672192 [12957.279859] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [12968.780216] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [13005.455571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13005.663344] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13005.686651] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13005.742368] Lustre: lustre-MDT0000: new disk, initializing [13005.811896] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13005.829509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13010.056191] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13020.591878] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13020.699650] Lustre: 348827:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13020.745292] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13020.748795] Lustre: Skipped 1 previous similar message [13020.816380] Lustre: lustre-MDT0001: new disk, initializing [13020.933241] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13020.941341] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13025.278935] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13034.915060] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13041.595833] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13049.000425] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13049.502849] Lustre: lustre-OST0000: new disk, initializing [13049.508113] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13049.518166] Lustre: 350794:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13051.145966] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13051.165425] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13051.342242] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13058.068451] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13068.494068] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13074.054792] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13074.323432] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13079.280407] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13079.519348] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13082.085395] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13082.091371] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13082.100606] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13082.108989] Lustre: Skipped 1 previous similar message [13082.127215] Lustre: Skipped 1 previous similar message [13087.206782] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13087.224083] Lustre: Skipped 1 previous similar message [13087.821427] Lustre: server umount lustre-OST0000 complete [13092.321910] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13097.442540] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13097.451880] Lustre: Skipped 3 previous similar messages [13098.604857] Lustre: server umount lustre-MDT0000 complete [13102.570496] LustreError: 348834:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13102.611753] LustreError: 348834:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13102.982123] LustreError: 348820:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787594853 with bad export cookie 17974681443828556860 [13102.994411] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13102.998175] LustreError: 348820:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13103.028337] LustreError: 334670:0:(import.c:717:ptlrpc_connect_import_locked()) already connecting [13103.536704] Lustre: server umount lustre-MDT0001 complete [13112.769927] XFS (loop0): Unmounting Filesystem [13118.000254] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 14:07:46 (1787594866) [13125.659168] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13126.187460] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13126.204788] Lustre: Skipped 2 previous similar messages [13130.999846] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13140.145855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13145.363615] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13153.212526] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13159.096731] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13165.728548] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13172.225674] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13179.384460] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13191.649583] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13191.655192] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13191.668644] LustreError: Skipped 1 previous similar message [13191.669543] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13191.681163] Lustre: Skipped 2 previous similar messages [13191.692811] Lustre: Skipped 1 previous similar message [13194.821732] Lustre: server umount lustre-OST0000 complete [13201.890043] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13204.956882] Lustre: server umount lustre-MDT0000 complete [13207.010067] LustreError: 355261:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13207.036557] LustreError: 355261:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [13209.244161] LustreError: 355198:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787594960 with bad export cookie 17974681443828557742 [13209.251911] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13209.258355] LustreError: 355198:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [13209.690828] Lustre: server umount lustre-MDT0001 complete [13222.314382] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [13225.612060] Key type lgssc unregistered [13226.056695] LNet: 356987:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13226.069981] LNetError: 356987:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13226.091020] LNet: Removed LNI 192.168.204.131@tcp [13227.297156] Key type .llcrypt unregistered [13227.303958] Key type ._llcrypt unregistered [13267.577634] Key type ._llcrypt registered [13267.590165] Key type .llcrypt registered [13267.749699] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [13269.089613] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13269.138335] alg: No test for adler32 (adler32-zlib) [13270.207520] Lustre: Lustre: Build Version: 2.17.57_80_g8672ae7 [13270.444590] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [13272.167519] Key type lgssc registered [13273.335167] Lustre: Echo OBD driver; http://www.lustre.org/ [13282.346703] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13282.389095] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13283.821565] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13288.654741] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13294.050594] LustreError: 358559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13294.069873] LustreError: 358559:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13298.644534] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13299.199517] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13303.776836] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13312.064604] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13318.090654] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13324.441100] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13324.745744] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13330.152531] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13337.855182] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13348.320691] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13348.337342] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13348.339739] Lustre: Skipped 1 previous similar message [13348.354692] Lustre: Skipped 1 previous similar message [13350.073882] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.31@tcp (stopping) [13353.443492] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13353.451253] Lustre: Skipped 1 previous similar message [13354.392130] Lustre: server umount lustre-MDT0001 complete [13355.166556] LustreError: 358553:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.31@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13358.567704] LustreError: 358558:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13358.590563] LustreError: 358558:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13363.684806] LustreError: 358553:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13363.711386] LustreError: 358553:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13367.037421] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13367.238204] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13367.260171] Lustre: Found index 1 for lustre-MDT0001, updating log [13367.322831] Lustre: 358548:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13367.625894] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13372.417933] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13372.920644] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13377.019503] LustreError: 362158:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13377.031075] LustreError: 362158:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13377.045857] LustreError: 358551:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13377.147598] LustreError: 362161:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.131@tcp: cfg command failed: rc = -17 [13377.165176] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13377.165176] [13677.763413] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.31@tcp (stopping) [13680.097301] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13680.102624] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13682.842890] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.31@tcp (stopping) [13682.850368] Lustre: Skipped 2 previous similar messages [13683.946524] Lustre: server umount lustre-MDT0001 complete [13685.218336] LustreError: 363080:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13685.246488] LustreError: 363080:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13695.429561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13695.477943] LustreError: 358558:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13695.496803] LustreError: 358558:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [13695.566383] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13695.573642] Lustre: Found index 1 for lustre-MDT0001, updating log [13695.617595] Lustre: 358548:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13696.074527] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13700.545243] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13701.120701] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13704.692732] LustreError: 370333:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13704.703631] LustreError: 370333:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13704.722915] LustreError: 358551:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13704.727222] LustreError: 358551:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13704.824671] LustreError: 370336:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13704.832784] LustreError: 370336:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13704.836993] LustreError: 370336:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.131@tcp: cfg command failed: rc = -17 [13704.847157] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.131@tcp [13704.847157] [14008.303686] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [14008.314736] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14008.341527] Lustre: Skipped 1 previous similar message [14008.346403] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [14008.361388] Lustre: Skipped 1 previous similar message [14013.037938] Lustre: server umount lustre-MDT0001 complete [14013.413465] LustreError: 363080:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14013.452074] LustreError: 363080:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [14025.360425] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14025.489253] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [14025.493581] Lustre: Found index 1 for lustre-MDT0001, updating log [14025.551344] Lustre: 358546:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14025.932402] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [14031.352944] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [14031.392427] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [14033.921864] LustreError: 378296:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [14033.933899] LustreError: 378296:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [14033.947710] LustreError: 358551:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [14033.955845] LustreError: 358551:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [14034.070263] LustreError: 378310:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [14034.096127] LustreError: 378310:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.131@tcp: cfg command failed: rc = -114 [14034.112600] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [14034.112600] [14358.499856] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14358.520664] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14358.545657] Lustre: Skipped 1 previous similar message [14358.557477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14358.567133] Lustre: Skipped 1 previous similar message [14369.247206] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [14369.478540] Lustre: server umount lustre-MDT0000 complete [14373.189611] LustreError: 358541:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787596124 with bad export cookie 14795655282673577890 [14373.198448] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14373.202169] LustreError: 358541:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14374.374572] LustreError: 358846:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14374.374743] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14374.398597] LustreError: 358846:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [14374.421215] Lustre: Skipped 2 previous similar messages [14379.606560] Lustre: server umount lustre-MDT0001 complete [14383.038409] Lustre: server umount lustre-OST0000 complete [14392.342906] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [14402.405572] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [14436.517923] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 14:29:45 (1787596185) [14442.544440] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14442.755198] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14442.796348] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14442.855615] Lustre: lustre-MDT0000: new disk, initializing [14442.921172] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14442.941312] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14446.964295] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [14457.106798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14457.243857] Lustre: 388737:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14457.328901] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14457.334100] Lustre: Skipped 1 previous similar message [14457.416282] Lustre: lustre-MDT0001: new disk, initializing [14457.520801] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14457.553774] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14457.563767] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14461.677279] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [14471.123988] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14476.865376] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14483.788078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14484.073377] Lustre: lustre-OST0000: new disk, initializing [14484.076450] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14484.086212] Lustre: 390705:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14484.194389] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14490.899750] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [14494.325117] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14494.346931] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14494.524447] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14500.023931] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14509.030352] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [14509.030957] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14509.037920] LustreError: Skipped 1 previous similar message [14509.065266] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14509.077117] Lustre: Skipped 11 previous similar messages [14509.974519] Lustre: server umount lustre-MDT0000 complete [14510.050920] LustreError: 388749:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14510.066542] LustreError: 388749:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [14513.057658] LustreError: 388729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787596263 with bad export cookie 14795655282673579843 [14513.061931] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14513.065342] LustreError: 389542:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14513.067215] LustreError: 388729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [14513.393837] Lustre: server umount lustre-MDT0001 complete [14526.486293] Lustre: server umount lustre-OST0000 complete [14560.104572] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [14571.257328] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [14603.407586] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14307 sec ========= 14:32:31 (1787596351) [14604.951213] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 14:32:33 (1787596353) === [14608.417569] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 14:32:36 (1787596356) === [14644.189718] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [14647.023280] Key type lgssc unregistered [14647.363660] LNet: 397072:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14647.375884] LNetError: 397072:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14647.399225] LNet: Removed LNI 192.168.204.131@tcp [14648.407123] Key type .llcrypt unregistered [14648.410935] Key type ._llcrypt unregistered