[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 499769859 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003322] x2apic enabled [ 0.004010] Switched APIC routing to physical x2apic. [ 0.005015] kvm-guest: setup PV IPIs [ 0.008367] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010012] pid_max: default: 32768 minimum: 301 [ 0.011137] LSM: Security Framework initializing [ 0.013037] Yama: becoming mindful. [ 0.014038] SELinux: Initializing. [ 0.015079] *** VALIDATE selinux *** [ 0.023621] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027768] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028147] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030029] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031094] *** VALIDATE tmpfs *** [ 0.032470] *** VALIDATE proc *** [ 0.033250] *** VALIDATE cgroup *** [ 0.034012] *** VALIDATE cgroup2 *** [ 0.035264] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036170] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038030] Spectre V2 : User space: Vulnerable [ 0.039013] Speculative Store Bypass: Vulnerable [ 0.042638] debug: unmapping init [mem 0xffffffffb4259000-0xffffffffb4260fff] [ 0.044196] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045678] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046024] ... version: 2 [ 0.047011] ... bit width: 48 [ 0.048009] ... generic registers: 4 [ 0.049014] ... value mask: 0000ffffffffffff [ 0.050014] ... max period: 00007fffffffffff [ 0.051013] ... fixed-purpose events: 3 [ 0.052014] ... event mask: 000000070000000f [ 0.053311] rcu: Hierarchical SRCU implementation. [ 0.055608] smp: Bringing up secondary CPUs ... [ 0.056700] x86: Booting SMP configuration: [ 0.057040] .... node #0, CPUs: #1 #2 #3 [ 0.061016] smp: Brought up 1 node, 4 CPUs [ 0.063029] smpboot: Max logical packages: 1 [ 0.064014] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.207031] node 0 deferred pages initialised in 141ms [ 0.211063] devtmpfs: initialized [ 0.212340] x86/mm: Memory block size: 128MB [ 0.216078] gcov: version magic: 0x41383552 [ 0.219353] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.220084] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.224406] pinctrl core: initialized pinctrl subsystem [ 0.227266] [ 0.227817] ************************************************************* [ 0.230018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.233020] ** ** [ 0.236020] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.238017] ** ** [ 0.241019] ** This means that this kernel is built to expose internal ** [ 0.244018] ** IOMMU data structures, which may compromise security on ** [ 0.246015] ** your system. ** [ 0.250031] ** ** [ 0.252015] ** If you see this message and you are not debugging the ** [ 0.255021] ** kernel, report this immediately to your vendor! ** [ 0.257018] ** ** [ 0.260019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.263017] ************************************************************* [ 0.266824] NET: Registered protocol family 16 [ 0.269512] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.272097] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.276086] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.280156] cpuidle: using governor menu [ 0.282720] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.285656] PCI: Using configuration type 1 for base access [ 0.288140] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.298102] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.302024] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.306090] cryptd: max_cpu_qlen set to 1000 [ 0.311252] ACPI: Added _OSI(Module Device) [ 0.313019] ACPI: Added _OSI(Processor Device) [ 0.315021] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.317017] ACPI: Added _OSI(Processor Aggregator Device) [ 0.321710] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.331298] ACPI: Interpreter enabled [ 0.333065] ACPI: PM: (supports S0 S3 S4 S5) [ 0.334019] ACPI: Using IOAPIC for interrupt routing [ 0.335170] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.338586] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.349000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.351075] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.354029] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.358086] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.364662] acpiphp: Slot [2] registered [ 0.366223] acpiphp: Slot [5] registered [ 0.368273] acpiphp: Slot [6] registered [ 0.369226] acpiphp: Slot [7] registered [ 0.371155] acpiphp: Slot [8] registered [ 0.373129] acpiphp: Slot [9] registered [ 0.374211] acpiphp: Slot [10] registered [ 0.376222] acpiphp: Slot [3] registered [ 0.378202] acpiphp: Slot [4] registered [ 0.379130] acpiphp: Slot [11] registered [ 0.380109] acpiphp: Slot [12] registered [ 0.382121] acpiphp: Slot [13] registered [ 0.386140] acpiphp: Slot [14] registered [ 0.387147] acpiphp: Slot [15] registered [ 0.389134] acpiphp: Slot [16] registered [ 0.390166] acpiphp: Slot [17] registered [ 0.392137] acpiphp: Slot [18] registered [ 0.394141] acpiphp: Slot [19] registered [ 0.396153] acpiphp: Slot [20] registered [ 0.398151] acpiphp: Slot [21] registered [ 0.399176] acpiphp: Slot [22] registered [ 0.401216] acpiphp: Slot [23] registered [ 0.402356] acpiphp: Slot [24] registered [ 0.404161] acpiphp: Slot [25] registered [ 0.406134] acpiphp: Slot [26] registered [ 0.408159] acpiphp: Slot [27] registered [ 0.409133] acpiphp: Slot [28] registered [ 0.410082] acpiphp: Slot [29] registered [ 0.411143] acpiphp: Slot [30] registered [ 0.413110] acpiphp: Slot [31] registered [ 0.414086] PCI host bridge to bus 0000:00 [ 0.416024] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.418029] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.419030] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.422042] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.425017] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.428042] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.430179] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.434085] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.437472] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.447779] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.453999] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.458034] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.460022] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.465027] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.468640] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.470682] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.473051] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.476989] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.482023] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.496024] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.502735] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.507949] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.515019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.520019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.540020] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.554833] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.565021] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.571017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.594021] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.607000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.619021] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.627017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.645016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.655265] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.667015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.672015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.691013] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.700973] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.710014] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.722023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.739015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.747680] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.754013] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.759015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.788016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.800730] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.804437] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.806394] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.809507] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.812201] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.817070] iommu: Default domain type: Passthrough [ 0.819373] SCSI subsystem initialized [ 0.820149] ACPI: bus type USB registered [ 0.822122] usbcore: registered new interface driver usbfs [ 0.824084] usbcore: registered new interface driver hub [ 0.826079] usbcore: registered new device driver usb [ 0.828159] pps_core: LinuxPPS API ver. 1 registered [ 0.830010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.833070] PTP clock support registered [ 0.836065] EDAC MC: Ver: 3.0.0 [ 0.838128] PCI: Using ACPI for IRQ routing [ 0.841139] NetLabel: Initializing [ 0.842024] NetLabel: domain hash size = 128 [ 0.844014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.847103] NetLabel: unlabeled traffic allowed by default [ 0.849230] vgaarb: loaded [ 0.851297] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.852014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.858001] clocksource: Switched to clocksource kvm-clock [ 0.966437] VFS: Disk quotas dquot_6.6.0 [ 0.968255] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.971172] *** VALIDATE ramfs *** [ 0.972421] *** VALIDATE hugetlbfs *** [ 0.974083] pnp: PnP ACPI init [ 0.976590] pnp: PnP ACPI: found 6 devices [ 0.996352] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.001366] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.004798] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.007861] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.010847] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.013789] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.017190] NET: Registered protocol family 2 [ 1.020125] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.025575] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.028616] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.034098] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.038284] TCP: Hash tables configured (established 65536 bind 65536) [ 1.042072] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.045870] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.049367] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.053016] NET: Registered protocol family 1 [ 1.056080] RPC: Registered named UNIX socket transport module. [ 1.059107] RPC: Registered udp transport module. [ 1.060905] RPC: Registered tcp transport module. [ 1.062741] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.065646] NET: Registered protocol family 44 [ 1.067537] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.069920] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.071793] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.074139] PCI: CLS 0 bytes, default 64 [ 1.076255] Unpacking initramfs... [ 2.522288] debug: unmapping init [mem 0xffff97057cc54000-0xffff97057ffbffff] [ 2.526860] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.528881] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.531522] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.036323] Initialise system trusted keyrings [ 3.037898] Key type blacklist registered [ 3.039611] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.052769] zbud: loaded [ 3.056736] *** VALIDATE nfs *** [ 3.057817] *** VALIDATE nfs4 *** [ 3.059774] pstore: using deflate compression [ 3.063191] Platform Keyring initialized [ 3.164278] NET: Registered protocol family 38 [ 3.166449] Key type asymmetric registered [ 3.168377] Asymmetric key parser 'x509' registered [ 3.170786] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.173579] io scheduler mq-deadline registered [ 3.175166] io scheduler kyber registered [ 3.176877] io scheduler bfq registered [ 3.179050] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.182030] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.184707] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.187498] ACPI: Power Button [PWRF] [ 3.193053] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.199768] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.211975] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.218689] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.232744] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.262443] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.289713] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.294786] Non-volatile memory driver v1.3 [ 3.296733] Linux agpgart interface v0.103 [ 3.329257] virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.332411] vda: detected capacity change from 0 to 74702848 [ 3.346699] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.349678] vdb: detected capacity change from 0 to 1073741824 [ 3.366671] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.370846] vdc: detected capacity change from 0 to 2621440000 [ 3.389371] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.392624] vdd: detected capacity change from 0 to 2621440000 [ 3.408972] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.412135] vde: detected capacity change from 0 to 4294967296 [ 3.428819] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.431891] vdf: detected capacity change from 0 to 4294967296 [ 3.439203] libphy: Fixed MDIO Bus: probed [ 3.443839] usbcore: registered new interface driver usbserial_generic [ 3.446690] usbserial: USB Serial support registered for generic [ 3.449409] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.454995] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.456589] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.459966] mousedev: PS/2 mouse device common for all mice [ 3.463555] rtc_cmos 00:05: RTC can wake from S4 [ 3.466372] rtc_cmos 00:05: registered as rtc0 [ 3.467789] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.468359] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.474320] intel_pstate: CPU model not supported [ 3.477029] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.478262] hid: raw HID events driver (C) Jiri Kosina [ 3.486077] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.486404] usbcore: registered new interface driver usbhid [ 3.490314] usbhid: USB HID core driver [ 3.491910] drop_monitor: Initializing network drop monitor service [ 3.494632] Initializing XFRM netlink socket [ 3.497093] NET: Registered protocol family 10 [ 3.500327] Segment Routing with IPv6 [ 3.501490] NET: Registered protocol family 17 [ 3.503963] mpls_gso: MPLS GSO support [ 3.509602] RAS: Correctable Errors collector initialized. [ 3.511527] AVX version of gcm_enc/dec engaged. [ 3.513135] AES CTR mode by8 optimization enabled [ 3.609708] sched_clock: Marking stable (3609687428, 0)->(4546420109, -936732681) [ 3.613718] registered taskstats version 1 [ 3.616425] Loading compiled-in X.509 certificates [ 3.618326] zswap: loaded using pool lzo/zbud [ 3.644930] Key type big_key registered [ 3.660169] Key type encrypted registered [ 3.661479] ima: No TPM chip found, activating TPM-bypass! [ 3.663824] ima: Allocated hash algorithm: sha1 [ 3.667654] ima: No architecture policies found [ 3.669668] evm: Initialising EVM extended attributes: [ 3.671794] evm: security.selinux [ 3.673162] evm: security.ima [ 3.674254] evm: security.capability [ 3.675408] evm: HMAC attrs: 0x1 [ 3.677603] rtc_cmos 00:05: setting system clock to 2026-08-15 16:42:58 UTC (1786812178) [ 3.684324] debug: unmapping init [mem 0xffffffffb5203000-0xffffffffb53fffff] [ 3.688069] debug: unmapping init [mem 0xffffffffb3f82000-0xffffffffb4258fff] [ 3.702096] Write protecting the kernel read-only data: 28672k [ 3.705759] debug: unmapping init [mem 0xffffffffb2603000-0xffffffffb27fffff] [ 3.708655] debug: unmapping init [mem 0xffffffffb2f14000-0xffffffffb2ffffff] [ 3.748384] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.759532] systemd[1]: Detected virtualization kvm. [ 3.761564] systemd[1]: Detected architecture x86-64. [ 3.763725] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.792337] systemd[1]: No hostname configured. [ 3.793979] systemd[1]: Set hostname to . [ 3.796182] random: systemd: uninitialized urandom read (16 bytes read) [ 3.799528] systemd[1]: Initializing machine ID from random generator. [ 3.939113] random: systemd: uninitialized urandom read (16 bytes read) [ 3.942799] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.947626] random: systemd: uninitialized urandom read (16 bytes read) [ 3.951060] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.955749] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Swap. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Timers. Starting Create Volatile Files and Directories... [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Reached target Local Encrypted Volumes. Starting Journal Service... [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.698224] device-mapper: uevent: version 1.0.3 [ 4.701146] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.576738] virtio_net virtio0 ens2: renamed from eth0 [ 5.588400] random: fast init done [ 5.670669] scsi host0: ata_piix [ 5.687286] scsi host1: ata_piix [ 5.689222] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.692232] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.545579] dracut-initqueue[581]: RTNETLINK answers: File exists [ 10.181648] random: crng init done [ 10.183173] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.904594] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.216659] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.507931] SELinux: Disabled at runtime. [ 12.565416] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.573618] systemd[1]: Detected virtualization kvm. [ 12.576056] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.105545] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.108537] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.112818] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.116398] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.119568] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.126185] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.130641] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File System... [ OK ] Created slice User and Session Slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ 13.189788] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... Mounting Huge Pages File System... [ OK ] Reached target RPC Port Mapper. Starting udev Coldplug all Devices... Starting Remount Root and Kernel File Systems... [ OK ] Listening on initctl Compatibility Named Pipe. Mounting POSIX Message Queue File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 13.656816] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.979086] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.014951] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.134397] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.168416] EDAC sbridge: Ver: 1.1.2 [ 16.062289] Key type dns_resolver registered [ 16.382487] NFS: Registering the id_resolver key type [ 16.384844] Key type id_resolver registered [ 16.386410] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started RPC Bind. [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg614-server login: [ 54.336695] hrtimer: interrupt took 5684636 ns [ 68.307905] libcfs: loading out-of-tree module taints kernel. [ 68.425383] Key type ._llcrypt registered [ 68.429701] Key type .llcrypt registered [ 68.581425] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 88.971574] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 90.506463] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 90.525632] alg: No test for adler32 (adler32-zlib) [ 91.909826] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 92.674576] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 94.359653] Key type lgssc registered [ 95.771921] Lustre: Echo OBD driver; http://www.lustre.org/ [ 113.907362] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 159.020140] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 172.878931] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 172.924971] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 174.105551] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 174.151455] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 174.253972] Lustre: lustre-MDT0000: new disk, initializing [ 174.361995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 174.398128] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 178.512508] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 192.942852] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 193.170785] Lustre: 6510:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 193.220452] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 193.227079] Lustre: Skipped 1 previous similar message [ 193.371741] Lustre: lustre-MDT0001: new disk, initializing [ 193.467782] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 193.506333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 193.533243] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 198.035495] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 202.548270] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 211.318789] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 211.528872] Lustre: lustre-OST0000: new disk, initializing [ 211.532562] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 211.537170] Lustre: 8448:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 211.583301] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 213.558201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 213.565603] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 213.652705] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 218.522455] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 233.258636] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 233.447576] Lustre: lustre-OST0001: new disk, initializing [ 233.450479] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 233.454722] Lustre: 9523:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 233.520199] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 239.154253] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 243.273178] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 243.291252] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 243.378489] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 250.579289] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 256.852192] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 263.323543] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing check_logdir /tmp/testlogs/ [ 269.457333] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing yml_node [ 273.774672] Lustre: DEBUG MARKER: Client: 2.17.57.3 [ 276.317192] Lustre: DEBUG MARKER: MDS: 2.17.57.3 [ 279.107885] Lustre: DEBUG MARKER: OSS: 2.17.57.3 [ 280.743038] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Aug 15 12:47:33 EDT 2026 [ 297.812913] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 299.399468] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 315.367429] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 315.371973] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 315.384093] Lustre: Skipped 2 previous similar messages [ 315.402080] Lustre: Skipped 1 previous similar message [ 320.481142] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 320.485368] Lustre: Skipped 3 previous similar messages [ 320.948682] Lustre: server umount lustre-MDT0000 complete [ 324.466529] LustreError: 6503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786812499 with bad export cookie 2984747186113322657 [ 324.474561] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 324.478825] LustreError: 6503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 325.606355] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 325.618392] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 325.628447] Lustre: Skipped 2 previous similar messages [ 329.632297] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 329.643566] Lustre: Skipped 2 previous similar messages [ 330.880696] Lustre: server umount lustre-MDT0001 complete [ 334.625849] Lustre: server umount lustre-OST0000 complete [ 338.296306] Lustre: server umount lustre-OST0001 complete [ 344.034650] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 352.557741] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 382.834726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 383.078927] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 383.116094] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 383.174865] Lustre: lustre-MDT0000: new disk, initializing [ 383.237846] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 383.259874] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 387.393625] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 397.478348] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 397.627167] Lustre: 15072:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 397.671483] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 397.674027] Lustre: Skipped 1 previous similar message [ 397.730856] Lustre: lustre-MDT0001: new disk, initializing [ 397.799518] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 397.844218] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 397.858429] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 402.493790] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 411.610979] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 417.466190] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 423.458968] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 423.711165] Lustre: lustre-OST0000: new disk, initializing [ 423.717980] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 423.726456] Lustre: 17043:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 423.813870] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 425.095434] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 425.113895] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 425.268529] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 429.679868] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 439.813698] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 444.779245] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 445.106473] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 450.815441] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 451.080112] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 456.160330] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 456.164173] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 456.168724] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 456.184850] Lustre: Skipped 1 previous similar message [ 456.206076] Lustre: Skipped 1 previous similar message [ 459.158414] Lustre: server umount lustre-OST0000 complete [ 464.353075] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 464.367694] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 464.394691] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 470.764965] Lustre: server umount lustre-MDT0000 complete [ 471.524363] LustreError: 15084:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 471.533640] LustreError: 15084:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 474.293358] LustreError: 17042:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786812649 with bad export cookie 2984747186113324183 [ 474.312093] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 474.316825] LustreError: 17042:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 474.654277] Lustre: server umount lustre-MDT0001 complete [ 480.637467] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 12:50:52 (1786812652) [ 488.132568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 488.733501] LustreError: 19308:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 488.782983] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 492.984752] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 494.051966] LustreError: 19309:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 499.169323] LustreError: 19308:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 502.539452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 507.711243] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 515.222854] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 520.290183] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 526.838902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 532.323920] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 540.518328] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 544.480638] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 553.969688] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 553.976265] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 553.978450] Lustre: Skipped 3 previous similar messages [ 553.992164] Lustre: Skipped 3 previous similar messages [ 556.655521] Lustre: server umount lustre-MDT0000 complete [ 559.072435] LustreError: 19309:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 559.087260] LustreError: 19309:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 560.460877] LustreError: 19290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786812735 with bad export cookie 2984747186113325072 [ 560.461911] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 560.472860] LustreError: 19290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 560.872984] Lustre: server umount lustre-MDT0001 complete [ 575.916197] Lustre: server umount lustre-OST0000 complete [ 596.953443] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 607.282541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 607.690148] LustreError: 23688:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 607.778687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 607.787031] Lustre: Skipped 2 previous similar messages [ 611.981447] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 619.630724] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 623.788300] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 626.215244] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 631.619881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 637.491953] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 645.188062] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 645.333530] Lustre: lustre-OST0001: new disk, initializing [ 645.336527] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 645.341687] Lustre: 26003:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 646.781908] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 646.809826] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 646.867222] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 651.086840] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 661.177509] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 664.957611] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 681.442440] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 681.458941] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 681.494069] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 681.505850] Lustre: Skipped 1 previous similar message [ 686.079371] Lustre: server umount lustre-MDT0000 complete [ 688.100483] LustreError: 23688:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 688.125545] LustreError: 23688:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 690.550749] LustreError: 23668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786812865 with bad export cookie 2984747186113325919 [ 690.558200] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 690.566734] LustreError: 23668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 691.072756] Lustre: server umount lustre-MDT0001 complete [ 705.116417] Lustre: server umount lustre-OST0000 complete [ 718.446724] Lustre: server umount lustre-OST0001 complete [ 734.294247] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 743.080993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 743.507854] LustreError: 29027:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 743.642080] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 743.649891] Lustre: Skipped 3 previous similar messages [ 747.716653] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 757.098846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 762.009925] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 766.077958] Lustre: 30175:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 773.269669] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 780.821049] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 789.228744] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 796.678496] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 805.580604] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 810.175361] Lustre: 32054:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 819.176760] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 819.192662] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 819.230513] Lustre: Skipped 3 previous similar messages [ 819.248374] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 819.261605] Lustre: Skipped 4 previous similar messages [ 823.325601] Lustre: server umount lustre-MDT0000 complete [ 825.323771] LustreError: 29027:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 825.361605] LustreError: 29027:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 826.761306] LustreError: 29009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786813001 with bad export cookie 2984747186113327214 [ 826.779164] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 826.783717] LustreError: 29009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 827.233464] Lustre: server umount lustre-MDT0001 complete [ 842.630965] Lustre: server umount lustre-OST0000 complete [ 846.818131] Lustre: 3643:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786813005/real 1786813005] req@ffff9705ffa68700 x1873608459973248/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786813021 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 847.988629] Lustre: server umount lustre-OST0001 complete [ 859.937992] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 12:57:12 (1786813032) [ 891.865188] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 900.217601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 905.235582] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 914.630414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 919.306002] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 922.969904] Lustre: 36007:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 931.828868] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 938.521508] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 946.425763] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 952.195657] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 958.712711] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 961.663433] Lustre: 37887:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 966.112563] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 966.117586] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 966.125230] Lustre: Skipped 4 previous similar messages [ 966.128290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 966.135604] Lustre: Skipped 4 previous similar messages [ 971.926523] Lustre: server umount lustre-MDT0000 complete [ 972.258047] LustreError: 34856:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 972.283931] LustreError: 34856:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 975.263990] LustreError: 35609:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786813150 with bad export cookie 2984747186113328313 [ 975.269499] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 975.272738] LustreError: 35609:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 975.532366] Lustre: server umount lustre-MDT0001 complete [ 989.360518] Lustre: server umount lustre-OST0000 complete [ 1002.482934] Lustre: server umount lustre-OST0001 complete [ 1010.553753] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 12:59:43 (1786813183) [ 1027.263155] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 1037.909254] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1038.559732] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1038.568705] Lustre: Skipped 7 previous similar messages [ 1042.187761] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1049.965545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1053.844842] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1056.411769] Lustre: 41180:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1062.079211] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1068.436266] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1078.214880] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1089.302773] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1097.955475] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1101.825925] Lustre: 43069:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1103.098231] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1103.106449] Lustre: Skipped 1 previous similar message [ 1106.407196] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1106.411735] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1106.424677] Lustre: Skipped 3 previous similar messages [ 1112.401737] Lustre: server umount lustre-MDT0000 complete [ 1115.719647] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786813290 with bad export cookie 2984747186113329419 [ 1115.729436] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1115.731316] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1122.085877] Lustre: server umount lustre-MDT0001 complete [ 1129.746666] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1140.191970] LustreError: 3641:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9705ffa6b100 x1873608460076288/t0(0) o250->MGC192.168.206.114@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1148.826944] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1150.414457] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1161.778302] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1167.354265] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1167.362792] Lustre: Skipped 1 previous similar message [ 1167.462611] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1175.271592] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1176.911661] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1182.727447] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1184.335450] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1192.930957] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1195.153734] Lustre: server umount lustre-MDT0000 complete [ 1199.151699] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786813373 with bad export cookie 2984747186113330532 [ 1199.155284] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1199.168309] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1199.576646] Lustre: server umount lustre-MDT0001 complete [ 1215.678554] Lustre: server umount lustre-OST0000 complete [ 1219.552543] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786813378/real 1786813378] req@ffff9705c9119880 x1873608460113664/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786813394 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1219.906255] Lustre: server umount lustre-OST0001 complete [ 1230.426422] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 13:03:22 (1786813402) [ 1247.243594] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 1258.587476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1258.862586] LustreError: 47674:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1258.878456] LustreError: 47674:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 37 previous similar messages [ 1262.920853] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1271.120031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1275.651913] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1278.811638] Lustre: 48826:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1285.167557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1292.424743] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1302.255869] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1310.462184] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1319.827968] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1326.585370] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1344.490548] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1344.498625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1344.514492] Lustre: Skipped 12 previous similar messages [ 1344.533426] Lustre: Skipped 20 previous similar messages [ 1350.731928] Lustre: server umount lustre-MDT0000 complete [ 1356.491432] LustreError: 47654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786813531 with bad export cookie 2984747186113331351 [ 1356.499449] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1356.515395] LustreError: 47654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1357.103670] Lustre: server umount lustre-MDT0001 complete [ 1371.978971] Lustre: server umount lustre-OST0000 complete [ 1375.200409] Lustre: 3643:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786813534/real 1786813534] req@ffff9705fb5d8a80 x1873608460174080/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786813550 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1377.152366] Lustre: server umount lustre-OST0001 complete [ 1391.271800] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 13:06:03 (1786813563) [ 1394.168063] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1396.157291] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 13:06:08 (1786813568) [ 1405.464141] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 1415.182916] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 1437.866240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1438.083932] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1438.204816] Lustre: lustre-MDT0000: new disk, initializing [ 1438.289276] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1441.742887] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1451.574613] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1459.975573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1460.283828] Lustre: lustre-OST0000: new disk, initializing [ 1460.289581] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1460.296561] Lustre: Skipped 1 previous similar message [ 1460.307035] Lustre: 54923:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1461.632024] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1461.650559] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1461.703058] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1468.365963] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1479.042453] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1481.764665] Lustre: DEBUG MARKER: create test files [ 1512.940460] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1516.848747] Lustre: server umount lustre-OST0000 complete [ 1520.793488] Lustre: server umount lustre-MDT0000 complete [ 1530.437514] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 1533.720902] Key type lgssc unregistered [ 1533.965947] LNet: 56555:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1533.974456] LNetError: 56555:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1535.016528] LNet: Removed LNI 192.168.206.114@tcp [ 1536.120556] Key type .llcrypt unregistered [ 1536.126107] Key type ._llcrypt unregistered [ 1566.453318] Key type ._llcrypt registered [ 1566.457892] Key type .llcrypt registered [ 1566.645442] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 1568.337331] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1568.394417] alg: No test for adler32 (adler32-zlib) [ 1569.817601] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 1570.130642] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 1571.943274] Key type lgssc registered [ 1573.447396] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1583.013955] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1583.046858] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1584.559682] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1588.887860] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1598.750465] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1605.742343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1606.102930] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1609.211690] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1609.295718] Lustre: 59295:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1609.306437] Lustre: 59295:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1609.320194] Lustre: 59295:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1609.330708] Lustre: 59295:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1609.340186] Lustre: 59295:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1609.350869] Lustre: 59295:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1611.791042] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1619.607077] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1626.225376] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1632.012037] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1639.904712] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1639.915645] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1639.931639] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1641.600633] Lustre: server umount lustre-OST0000 complete [ 1645.451741] Lustre: server umount lustre-MDT0000 complete [ 1654.723811] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 1657.197603] Key type lgssc unregistered [ 1657.450967] LNet: 60909:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1657.458129] LNetError: 60909:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1657.468602] LNet: Removed LNI 192.168.206.114@tcp [ 1658.205580] Key type .llcrypt unregistered [ 1658.210634] Key type ._llcrypt unregistered [ 1691.254877] Key type ._llcrypt registered [ 1691.257224] Key type .llcrypt registered [ 1691.363417] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 1692.670464] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1692.734095] alg: No test for adler32 (adler32-zlib) [ 1693.893911] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 1694.134320] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 1695.799744] Key type lgssc registered [ 1697.154479] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1704.423922] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1704.458522] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1705.908294] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1710.399291] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1717.392739] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1723.635906] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1723.967220] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1729.154974] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1732.139518] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1732.207899] Lustre: 63742:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1732.212599] Lustre: 63742:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1732.216856] Lustre: 63742:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1732.220328] Lustre: 63742:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1732.224638] Lustre: 63742:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1732.229961] Lustre: 63742:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1735.421679] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1740.173875] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1747.426645] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1747.426798] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1747.446326] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1752.545169] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1753.495156] Lustre: server umount lustre-OST0000 complete [ 1756.837944] Lustre: server umount lustre-MDT0000 complete [ 1766.001872] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 1768.389880] Key type lgssc unregistered [ 1768.664632] LNet: 65303:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1768.674696] LNetError: 65303:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1768.691877] LNet: Removed LNI 192.168.206.114@tcp [ 1769.425757] Key type .llcrypt unregistered [ 1769.428402] Key type ._llcrypt unregistered [ 1785.508540] Key type ._llcrypt registered [ 1785.509965] Key type .llcrypt registered [ 1785.617251] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 1796.773505] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 1797.717378] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1797.736950] alg: No test for adler32 (adler32-zlib) [ 1798.698165] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 1798.881505] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 1800.511258] Key type lgssc registered [ 1801.456659] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1829.613458] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1829.630066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1830.782654] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1830.806982] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1830.869278] Lustre: lustre-MDT0000: new disk, initializing [ 1830.910580] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1830.929743] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1834.633568] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1843.541895] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1843.615511] Lustre: 68431:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1843.658845] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1843.662209] Lustre: Skipped 1 previous similar message [ 1843.713168] Lustre: lustre-MDT0001: new disk, initializing [ 1843.787737] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1843.825391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1843.832906] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1846.961799] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1854.818969] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1858.953845] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1863.384752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1863.639658] Lustre: lustre-OST0000: new disk, initializing [ 1863.642767] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1863.647625] Lustre: 70402:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1863.714324] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1868.345861] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1871.917195] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1871.924734] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1872.016643] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1875.770360] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1880.243610] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1880.440882] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1884.959605] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1885.111875] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1887.201620] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1887.205021] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1887.228763] Lustre: Skipped 1 previous similar message [ 1892.331111] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1892.342529] Lustre: Skipped 2 previous similar messages [ 1892.787926] Lustre: server umount lustre-OST0000 complete [ 1897.447536] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1897.452331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1897.475658] Lustre: Skipped 1 previous similar message [ 1901.837602] Lustre: server umount lustre-MDT0000 complete [ 1904.993277] LustreError: 68425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786814079 with bad export cookie 5130459466664510604 [ 1904.997869] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1905.001500] LustreError: 68425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1905.270423] Lustre: server umount lustre-MDT0001 complete [ 1911.956778] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 13:14:45 (1786814085) [ 1919.774799] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 1946.194472] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 1952.227924] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 1975.300993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1975.483628] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1975.500673] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1975.550599] Lustre: lustre-MDT0000: new disk, initializing [ 1975.595376] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1975.608707] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1978.626382] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1986.078433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1986.170781] Lustre: 76417:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1986.225405] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1986.230978] Lustre: Skipped 1 previous similar message [ 1986.293681] Lustre: lustre-MDT0001: new disk, initializing [ 1986.351061] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1986.369363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1986.375377] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1989.203366] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 1995.436670] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1999.583594] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2005.378818] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2005.654177] Lustre: lustre-OST0000: new disk, initializing [ 2005.663978] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2005.675286] Lustre: 78384:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2005.761224] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2011.076202] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2014.761967] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2014.771423] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2014.822868] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2018.621229] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2022.574212] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2022.713884] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2025.979940] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2026.180521] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2030.052391] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2030.052741] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2030.062122] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2030.074904] Lustre: Skipped 1 previous similar message [ 2030.085823] Lustre: Skipped 2 previous similar messages [ 2033.802880] Lustre: server umount lustre-OST0000 complete [ 2037.730231] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2037.734625] LustreError: Skipped 1 previous similar message [ 2037.737797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2037.750152] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2042.641468] Lustre: server umount lustre-MDT0000 complete [ 2045.392765] LustreError: 76407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786814220 with bad export cookie 5130459466664511486 [ 2045.393538] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2045.404524] LustreError: 76407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2045.419858] LustreError: 76428:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2045.429864] LustreError: 76428:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2045.623972] Lustre: server umount lustre-MDT0001 complete [ 2050.970907] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 13:17:04 (1786814224) [ 2055.220624] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2055.470159] LustreError: 80794:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2055.521920] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2058.466642] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2060.768152] LustreError: 80795:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2064.514711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2068.199737] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2075.576412] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2081.330356] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2087.438713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2087.658862] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2087.662344] Lustre: Skipped 1 previous similar message [ 2092.646757] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2099.302078] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2103.750944] Lustre: 83462:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting d7eb6924-793c-49ba-b7dc-dba92a9effed at adminstrative request [ 2106.344070] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2106.364271] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2116.582335] Lustre: MGS: Client d7eb6924-793c-49ba-b7dc-dba92a9effed (at 0@lo) reconnecting [ 2116.601051] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x47330db3dd140b70 to 0x47330db3dd140e10 [ 2116.626160] Lustre: MGC192.168.206.114@tcp: Connection restored to 0@lo (at 0@lo) [ 2141.498449] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2141.654773] Lustre: lustre-OST0001: new disk, initializing [ 2141.658837] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2141.667377] Lustre: 83860:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2147.645980] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2150.954940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2150.965842] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2151.030181] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2155.665918] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2161.122597] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2161.125567] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2161.128919] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2161.128927] Lustre: Skipped 2 previous similar messages [ 2161.160777] Lustre: Skipped 2 previous similar messages [ 2164.936767] Lustre: server umount lustre-OST0001 complete [ 2166.242603] LustreError: 84580:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2166.275168] LustreError: 84580:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2171.378120] LustreError: 83476:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2171.381857] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2171.404166] LustreError: 83476:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2171.439552] Lustre: Skipped 1 previous similar message [ 2181.604598] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2181.607142] LustreError: 84580:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2181.627262] Lustre: Skipped 6 previous similar messages [ 2181.645699] LustreError: 84580:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2184.159191] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2184.438773] Lustre: server umount lustre-OST0000 complete [ 2191.840526] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2195.444743] Lustre: server umount lustre-MDT0000 complete [ 2200.622320] LustreError: 80776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786814375 with bad export cookie 5130459466664513040 [ 2200.628913] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2200.649708] LustreError: 80776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2201.329146] Lustre: server umount lustre-MDT0001 complete [ 2211.583706] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 13:19:44 (1786814384) [ 2213.071449] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2215.184612] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 13:19:47 (1786814387) [ 2216.461188] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2218.088968] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 13:19:50 (1786814390) [ 2219.868250] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2221.415474] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 13:19:54 (1786814394) [ 2223.006852] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2224.811682] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 13:19:57 (1786814397) [ 2226.211255] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2231.889907] loop: module loaded [ 2231.911166] loop0: detected capacity change from 0 to 4294967296 [ 2231.947100] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2233.308481] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2236.773178] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2240.271391] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2270.994131] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2271.074533] LustreError: 87985:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2271.082371] LustreError: 87985:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2271.095308] LustreError: 87985:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2271.110146] LustreError: 87985:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2289.788903] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 2304.679628] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 2349.937501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2350.133421] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2350.160960] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2350.259317] Lustre: lustre-MDT0000: new disk, initializing [ 2350.337405] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2350.342719] Lustre: Skipped 1 previous similar message [ 2350.360047] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2354.924650] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2365.252647] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2365.432630] Lustre: 90865:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2365.676283] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2370.648961] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2382.108319] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2388.242713] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2394.219935] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2394.555978] Lustre: 92833:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2399.783392] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2399.914760] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2400.910331] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2411.029737] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2415.079341] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2415.242618] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2419.603917] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2419.776348] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2425.312348] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2425.320567] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2425.338989] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2425.338997] Lustre: Skipped 2 previous similar messages [ 2425.403328] Lustre: Skipped 2 previous similar messages [ 2427.800345] Lustre: server umount lustre-OST0000 complete [ 2432.480849] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2446.303204] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2446.444665] Lustre: server umount lustre-MDT0000 complete [ 2450.923865] LustreError: 90876:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2450.970790] LustreError: 90876:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2451.261848] LustreError: 90859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786814626 with bad export cookie 5130459466664513649 [ 2451.284432] LustreError: 90859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2451.285322] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2451.593671] Lustre: server umount lustre-MDT0001 complete [ 2460.301387] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 13:23:52 (1786814632) [ 2467.481743] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2472.736339] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2482.213640] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2482.503177] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2482.511638] Lustre: Skipped 3 previous similar messages [ 2486.443618] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2493.972760] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2498.962458] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2505.108543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2511.565610] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2518.043488] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2522.275467] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2522.470870] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2526.419822] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2526.593779] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2532.432335] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2532.651730] Lustre: lustre-OST0001: new disk, initializing [ 2532.655956] Lustre: Skipped 2 previous similar messages [ 2532.661140] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2532.668628] Lustre: Skipped 3 previous similar messages [ 2532.678842] Lustre: 98619:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2537.822855] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2542.127298] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2542.135890] Lustre: Skipped 2 previous similar messages [ 2542.141349] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2542.215298] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2545.756967] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2550.082157] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2550.310192] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2554.189489] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2554.415605] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2573.587598] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2597.373359] Lustre: Failing over lustre-MDT0000 [ 2597.776283] Lustre: server umount lustre-MDT0000 complete [ 2598.822218] LustreError: 95245:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.206.14@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2598.863373] LustreError: 95245:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2598.887514] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2598.906598] Lustre: Skipped 5 previous similar messages [ 2614.240478] Lustre: 66659:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786814773/real 1786814773] req@00000000ea8b158e x1873610248809984/t0(0) o400->MGC192.168.206.114@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786814789 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2614.275882] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2621.308218] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2621.310098] LDISKFS-fs (dm-0): recovery complete [ 2621.319573] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2624.848942] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2628.954682] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2629.509370] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2629.509751] Lustre: 95246:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2630.126789] Lustre: 95250:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2630.137969] Lustre: 95250:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2630.153278] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2630.226489] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2630.551076] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2630.882743] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2630.893131] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2631.218471] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2631.228319] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2631.240495] Lustre: 101463:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2631.256612] Lustre: 101463:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2631.904551] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2631.916958] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2632.239276] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2632.247798] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2633.535551] Lustre: 101463:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2633.547355] Lustre: 101463:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2634.185416] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2634.198109] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2634.528882] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2634.534125] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2637.720811] Lustre: 101463:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2637.735018] Lustre: 101463:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2638.380330] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2638.387606] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2638.703120] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2638.715737] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2645.819305] Lustre: 101463:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2645.830194] Lustre: 101463:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 2646.475360] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2646.484829] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2646.807559] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2646.817861] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2662.028111] Lustre: 101463:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2662.043270] Lustre: 101463:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 2662.691529] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2662.701231] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2663.039219] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2663.047254] LustreError: 101463:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2689.079264] Lustre: 101463:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2689.088948] Lustre: 101463:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2689.096823] Lustre: 101463:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 78e2f553-c161-41d6-8d2b-31afce4ae524@192.168.206.14@tcp [ 2689.114537] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2689.127532] Lustre: 101463:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000a358e524 x1873610236041344/t0(8589934781) o101->78e2f553-c161-41d6-8d2b-31afce4ae524@192.168.206.14@tcp:164/0 lens 592/0 e 0 to 0 dl 1786814874 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2689.171420] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2689.194334] Lustre: lustre-MDT0000: Denying connection for new client 78e2f553-c161-41d6-8d2b-31afce4ae524 (at 192.168.206.14@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2689.211340] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2689.268549] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2689.272987] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2689.282045] Lustre: Skipped 2 previous similar messages [ 2689.348956] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2689.349761] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2705.377382] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2705.386079] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2705.392873] Lustre: Skipped 8 previous similar messages [ 2717.153100] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2717.232936] Lustre: server umount lustre-OST0000 complete [ 2721.760939] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2721.766794] LustreError: Skipped 1 previous similar message [ 2722.273598] LustreError: 97071:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2722.290767] LustreError: 97071:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 35 previous similar messages [ 2726.274706] Lustre: server umount lustre-OST0001 complete [ 2730.464565] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2730.484546] Lustre: Skipped 4 previous similar messages [ 2743.267307] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2743.470566] Lustre: server umount lustre-MDT0000 complete [ 2746.321209] LustreError: 95230:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786814921 with bad export cookie 5130459466664564875 [ 2746.325258] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2746.330216] LustreError: 95230:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2746.598035] Lustre: server umount lustre-MDT0001 complete [ 2754.059530] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 13:28:46 (1786814926) [ 2760.661894] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2760.668519] Lustre: *** cfs_fail_loc=197, val=0*** [ 2776.159579] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2781.279947] LustreError: 103441:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.114@tcp: failed processing log, type 4: rc = -110 [ 2786.491107] Lustre: *** cfs_fail_loc=197, val=0*** [ 2786.495330] Lustre: Skipped 65 previous similar messages [ 2807.074061] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2807.085945] Lustre: Skipped 3 previous similar messages [ 2813.133773] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2820.653330] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2822.693856] Lustre: server umount lustre-OST0000 complete [ 2831.339423] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 13:30:03 (1786815003) [ 2851.121452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2851.533750] LustreError: 105062:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2856.161376] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2865.761986] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2869.502822] LustreError: 105093:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2869.515689] LustreError: 105093:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2869.524151] LustreError: 105093:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 2869.826370] Lustre: server umount lustre-MDT0000 complete [ 2911.690531] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 2930.119845] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 2967.913711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2968.176894] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2968.199213] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2968.265302] Lustre: lustre-MDT0000: new disk, initializing [ 2968.356311] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2973.359476] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 2985.857061] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2986.029420] Lustre: 109295:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2986.339102] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2991.720592] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3003.293961] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3009.453721] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3016.884803] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3017.249159] Lustre: 111263:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3024.075102] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3025.607834] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3034.275681] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3039.496675] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3039.751183] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3044.885854] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3045.182743] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3050.976401] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3050.994508] LustreError: Skipped 1 previous similar message [ 3051.002303] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3051.024453] Lustre: Skipped 1 previous similar message [ 3051.032217] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3051.036493] Lustre: Skipped 17 previous similar messages [ 3053.498426] Lustre: server umount lustre-OST0000 complete [ 3063.901655] Lustre: server umount lustre-MDT0000 complete [ 3068.312356] LustreError: 109287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786815243 with bad export cookie 5130459466664575571 [ 3068.349743] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3068.653599] Lustre: server umount lustre-MDT0001 complete [ 3077.882044] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 13:34:10 (1786815250) [ 3086.958072] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 3089.667880] Key type lgssc unregistered [ 3089.893372] LNet: 113752:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3089.900489] LNetError: 113752:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3089.928194] LNet: Removed LNI 192.168.206.114@tcp [ 3090.991917] Key type .llcrypt unregistered [ 3090.998898] Key type ._llcrypt unregistered [ 3109.104902] Key type ._llcrypt registered [ 3109.107713] Key type .llcrypt registered [ 3109.219143] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 3127.232953] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 3128.255588] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3128.435339] alg: No test for adler32 (adler32-zlib) [ 3129.464494] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 3129.676990] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 3131.384415] Key type lgssc registered [ 3132.601838] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3177.299391] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3177.331714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3178.601238] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3178.645906] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3178.722639] Lustre: lustre-MDT0000: new disk, initializing [ 3178.850493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3178.871737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3183.735464] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3194.870522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3195.146786] Lustre: lustre-OST0000: new disk, initializing [ 3195.150333] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3195.154656] Lustre: Skipped 1 previous similar message [ 3195.163925] Lustre: 117925:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3195.245796] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3201.704510] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3203.696232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3203.706567] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3203.739538] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3211.370368] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3211.694122] Lustre: lustre-OST0001: new disk, initializing [ 3211.706243] Lustre: lustre-OST0001: Not available for connect from 0@lo (not set up) [ 3211.718754] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3211.735533] Lustre: 118793:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3211.841830] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3216.944055] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3216.951554] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3216.991919] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3218.364805] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3227.110115] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3227.119098] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3227.124808] Lustre: Skipped 1 previous similar message [ 3227.135727] Lustre: Skipped 1 previous similar message [ 3232.225746] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3232.232724] Lustre: Skipped 1 previous similar message [ 3233.443323] Lustre: server umount lustre-MDT0000 complete [ 3239.835385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3244.391495] LustreError: 117110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786815419 with bad export cookie 8992511023065968812 [ 3244.396491] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3244.402858] LustreError: 117110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3244.497735] Lustre: server umount lustre-OST0000 complete [ 3248.324127] Lustre: server umount lustre-OST0001 complete [ 3257.866484] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 13:37:10 (1786815430) [ 3281.827774] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 13:37:34 (1786815454) [ 3293.419715] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 3317.501973] Lustre: DEBUG MARKER: == conf-sanity test 89a: check tunefs --param and --erase-param{s} options ========================================================== 13:38:10 (1786815490) [ 3355.399461] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 3365.993949] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 3403.260121] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3403.707709] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3403.770256] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3403.885245] Lustre: lustre-MDT0000: new disk, initializing [ 3403.989170] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3404.011254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3408.779649] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3419.537516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3419.724494] Lustre: 126542:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3419.781583] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3419.785429] Lustre: Skipped 1 previous similar message [ 3419.865284] Lustre: lustre-MDT0001: new disk, initializing [ 3419.970057] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3419.997158] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3420.015035] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3424.463403] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3434.258122] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3439.910311] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3446.676556] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3447.206393] Lustre: lustre-OST0000: new disk, initializing [ 3447.210780] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3447.229983] Lustre: 128510:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3447.354678] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3454.551637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3454.563085] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3454.656536] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3454.724763] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3464.497686] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3469.559499] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3469.755969] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3474.863831] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3475.129620] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3480.544798] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3480.547022] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3480.559377] LustreError: Skipped 1 previous similar message [ 3480.561404] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3480.587813] Lustre: Skipped 1 previous similar message [ 3483.439556] Lustre: server umount lustre-OST0000 complete [ 3490.785829] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3490.803172] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3490.811662] Lustre: Skipped 2 previous similar messages [ 3494.047296] Lustre: server umount lustre-MDT0000 complete [ 3497.857574] LustreError: 127336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786815672 with bad export cookie 8992511023065970037 [ 3497.866860] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3497.868097] LustreError: 127336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3498.088277] Lustre: server umount lustre-MDT0001 complete [ 3507.405825] Lustre: DEBUG MARKER: == conf-sanity test 89b: tunefs --quota keeps the other options and creates quota inodes ========================================================== 13:41:19 (1786815679) [ 3546.929430] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 3559.485933] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 3598.678490] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3598.921335] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3598.953589] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3599.049476] Lustre: lustre-MDT0000: new disk, initializing [ 3599.127217] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3599.152783] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3603.289427] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3613.471849] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3613.604504] Lustre: 134368:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3613.866162] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3618.592871] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3628.610495] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3634.851105] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3641.298178] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3641.579801] Lustre: lustre-OST0000: new disk, initializing [ 3641.582545] Lustre: Skipped 1 previous similar message [ 3641.587893] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3641.593876] Lustre: Skipped 2 previous similar messages [ 3641.600475] Lustre: 136334:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3641.697620] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3641.708081] Lustre: Skipped 1 previous similar message [ 3647.412070] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3648.091083] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3648.103221] Lustre: Skipped 1 previous similar message [ 3648.107200] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3648.244322] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3658.822725] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3663.172670] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3663.387501] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3667.309035] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3667.417645] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3674.097985] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3674.098596] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3674.107067] LustreError: Skipped 1 previous similar message [ 3674.107815] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3674.137580] Lustre: Skipped 2 previous similar messages [ 3675.750710] Lustre: server umount lustre-OST0000 complete [ 3684.326758] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3684.337141] Lustre: Skipped 1 previous similar message [ 3687.372118] Lustre: server umount lustre-MDT0000 complete [ 3689.443882] LustreError: 137655:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3689.460958] LustreError: 137655:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 3690.807431] LustreError: 134360:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786815865 with bad export cookie 8992511023065970919 [ 3690.810713] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3690.820892] LustreError: 134360:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3691.209581] Lustre: server umount lustre-MDT0001 complete [ 3694.580895] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 13:44:27 (1786815867) [ 3700.311722] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3700.911505] LustreError: 138603:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3705.674612] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3706.336353] LustreError: 138604:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3706.362395] LustreError: 138604:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3710.435704] LustreError: 138603:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3714.020175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3714.454230] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3714.460720] Lustre: Skipped 1 previous similar message [ 3718.970785] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3726.044542] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3732.184553] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3739.067189] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3745.293538] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3752.957977] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3757.128101] Lustre: *** cfs_fail_loc=159, val=0*** [ 3773.386853] Lustre: lustre-MDT0000: Client 387c770d-12ca-432e-871e-a67331a6060e (at 192.168.206.14@tcp) reconnecting [ 3774.849263] Lustre: *** cfs_fail_loc=159, val=0*** [ 3774.861024] Lustre: Skipped 5 previous similar messages [ 3791.287556] Lustre: lustre-MDT0000: Client 387c770d-12ca-432e-871e-a67331a6060e (at 192.168.206.14@tcp) reconnecting [ 3812.327923] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3812.341711] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3812.351671] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3812.364711] Lustre: Skipped 4 previous similar messages [ 3814.509695] Lustre: server umount lustre-OST0000 complete [ 3822.052958] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3824.955582] Lustre: server umount lustre-MDT0000 complete [ 3827.684448] LustreError: 141259:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3827.722061] LustreError: 141259:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3828.487489] LustreError: 138583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786816003 with bad export cookie 8992511023065971801 [ 3828.500152] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3828.701901] Lustre: server umount lustre-MDT0001 complete [ 3838.969623] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 3841.436624] Key type lgssc unregistered [ 3841.728789] LNet: 142401:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3841.738444] LNetError: 142401:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3841.755930] LNet: Removed LNI 192.168.206.114@tcp [ 3842.456050] Key type .llcrypt unregistered [ 3842.458446] Key type ._llcrypt unregistered [ 3864.728230] Key type ._llcrypt registered [ 3864.732229] Key type .llcrypt registered [ 3864.866579] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 3865.703524] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3865.714915] alg: No test for adler32 (adler32-zlib) [ 3866.740291] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 3867.000243] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 3868.687210] Key type lgssc registered [ 3869.882577] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3877.167671] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3877.191683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3878.642894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3882.893882] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3889.128467] LustreError: 143815:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3889.159602] LustreError: 143815:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3891.577854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3891.931652] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3896.206995] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3903.988283] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3909.650231] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3915.870541] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3916.425830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3922.729896] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 3923.650694] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3931.039580] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3940.519507] Lustre: *** cfs_fail_loc=159, val=0*** [ 3956.179298] Lustre: lustre-MDT0000: Client af8a009b-51f8-4954-955c-42721977fa38 (at 192.168.206.14@tcp) reconnecting [ 3958.658673] Lustre: *** cfs_fail_loc=159, val=0*** [ 3974.070410] Lustre: lustre-MDT0001: Client af8a009b-51f8-4954-955c-42721977fa38 (at 192.168.206.14@tcp) reconnecting [ 3975.765040] Lustre: *** cfs_fail_loc=159, val=0*** [ 3975.766725] Lustre: Skipped 3 previous similar messages [ 3990.975384] Lustre: lustre-MDT0001: Client af8a009b-51f8-4954-955c-42721977fa38 (at 192.168.206.14@tcp) reconnecting [ 4009.080933] Lustre: 146877:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4012.535534] Lustre: *** cfs_fail_loc=159, val=0*** [ 4012.543260] Lustre: Skipped 5 previous similar messages [ 4028.320216] Lustre: lustre-MDT0000: Client 6690ee2d-1853-4590-a4d4-3c6b9c2faffc (at 192.168.206.14@tcp) reconnecting [ 4029.798532] Lustre: *** cfs_fail_loc=159, val=0*** [ 4029.803405] Lustre: Skipped 12 previous similar messages [ 4045.228904] Lustre: lustre-MDT0000: Client 6690ee2d-1853-4590-a4d4-3c6b9c2faffc (at 192.168.206.14@tcp) reconnecting [ 4046.636652] Lustre: 147169:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4066.275048] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4066.291954] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4066.317168] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4067.299979] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4067.300482] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4067.334395] Lustre: Skipped 1 previous similar message [ 4068.846710] Lustre: server umount lustre-OST0000 complete [ 4076.512240] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4076.526899] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4076.549316] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4079.044960] Lustre: server umount lustre-MDT0000 complete [ 4082.661870] LustreError: 146572:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4082.689232] LustreError: 146572:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4083.000404] LustreError: 143795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786816257 with bad export cookie 5542171096280732703 [ 4083.006313] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4083.010499] LustreError: 143795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4083.377256] Lustre: server umount lustre-MDT0001 complete [ 4094.310675] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 4096.660601] Key type lgssc unregistered [ 4096.928867] LNet: 148248:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4096.936435] LNetError: 148248:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4096.949063] LNet: Removed LNI 192.168.206.114@tcp [ 4097.839624] Key type .llcrypt unregistered [ 4097.842279] Key type ._llcrypt unregistered [ 4118.690025] Key type ._llcrypt registered [ 4118.692703] Key type .llcrypt registered [ 4118.801951] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 4119.773323] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4119.913463] alg: No test for adler32 (adler32-zlib) [ 4120.955479] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 4121.203461] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 4122.855393] Key type lgssc registered [ 4123.979879] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4131.260733] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4131.274354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4132.729274] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4137.373128] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4143.081465] LustreError: 149664:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4143.107774] LustreError: 149664:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4146.173996] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4146.456350] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4151.401741] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4159.393828] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4166.127747] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4172.438405] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4172.663983] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4177.454075] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4177.792962] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4184.990840] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4193.145853] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4194.946336] Lustre: 152246:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4197.480418] Lustre: 152294:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4198.782116] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4199.586387] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4203.703137] Lustre: 152441:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4204.829172] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4205.599491] Lustre: 152540:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4213.730578] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4213.733919] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4213.744327] LustreError: Skipped 1 previous similar message [ 4213.764102] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4213.773430] Lustre: Skipped 1 previous similar message [ 4216.274973] Lustre: server umount lustre-OST0000 complete [ 4223.456986] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4223.465398] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4223.478740] Lustre: Skipped 1 previous similar message [ 4223.485514] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4226.047589] Lustre: server umount lustre-MDT0000 complete [ 4229.091757] LustreError: 151172:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4229.108280] LustreError: 151172:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4229.745303] LustreError: 149644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786816404 with bad export cookie 18334014488608842510 [ 4229.746815] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4229.754981] LustreError: 149644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4230.141740] Lustre: server umount lustre-MDT0001 complete [ 4241.311470] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 4243.688970] Key type lgssc unregistered [ 4243.999622] LNet: 153605:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4244.009510] LNetError: 153605:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4244.032610] LNet: Removed LNI 192.168.206.114@tcp [ 4244.876597] Key type .llcrypt unregistered [ 4244.879144] Key type ._llcrypt unregistered [ 4268.001931] Key type ._llcrypt registered [ 4268.006646] Key type .llcrypt registered [ 4268.079506] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 4268.964498] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4269.024154] alg: No test for adler32 (adler32-zlib) [ 4270.130579] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 4270.338114] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 4271.992193] Key type lgssc registered [ 4272.932437] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4280.368460] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4280.407589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4281.957439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4286.268414] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4292.065841] LustreError: 155025:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4292.084993] LustreError: 155025:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4294.331861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4294.763034] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4299.801920] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4307.309935] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4313.508829] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4320.588500] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4321.055026] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4326.931286] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4327.218819] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4333.982688] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4339.518341] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4342.459706] Lustre: *** cfs_fail_loc=159, val=0*** [ 4358.068860] Lustre: lustre-MDT0000: Client c9f7e208-8b8c-4652-9989-c77eb9abd3ab (at 192.168.206.14@tcp) reconnecting [ 4375.521155] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4375.523022] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4375.552420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4375.557276] Lustre: Skipped 1 previous similar message [ 4380.643774] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4380.649520] Lustre: Skipped 1 previous similar message [ 4381.517407] Lustre: server umount lustre-OST0000 complete [ 4385.761303] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4385.772503] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4385.778837] Lustre: Skipped 2 previous similar messages [ 4390.881158] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4390.886901] Lustre: Skipped 1 previous similar message [ 4391.217764] Lustre: server umount lustre-MDT0000 complete [ 4394.654289] LustreError: 155008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786816569 with bad export cookie 14431556567354625259 [ 4394.663576] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4394.669740] LustreError: 155008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4394.919710] Lustre: server umount lustre-MDT0001 complete [ 4408.501347] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 4412.573342] Key type lgssc unregistered [ 4412.891503] LNet: 158693:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4412.900087] LNetError: 158693:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4412.933134] LNet: Removed LNI 192.168.206.114@tcp [ 4414.004473] Key type .llcrypt unregistered [ 4414.006749] Key type ._llcrypt unregistered [ 4438.151486] Key type ._llcrypt registered [ 4438.154457] Key type .llcrypt registered [ 4438.266218] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 4439.127375] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4439.266082] alg: No test for adler32 (adler32-zlib) [ 4440.455501] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 4440.715579] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 4442.424135] Key type lgssc registered [ 4443.985775] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4452.937558] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4452.973106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4454.540881] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4458.131138] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4465.122188] LustreError: 160109:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4465.151226] LustreError: 160109:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4467.581656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4468.040923] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4472.241343] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4480.973948] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4486.870670] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4493.764585] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4494.048332] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4500.110212] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4500.226607] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4507.180883] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4512.566407] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4530.552504] Lustre: lustre-MDT0001: evicting e53d70b2-f7bd-4e7b-ad35-6d83d8c8ac31 (at 192.168.206.14@tcp) by administrative request [ 4530.564973] Lustre: Skipped 1 previous similar message [ 4558.825672] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4558.830508] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4558.852168] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4563.938050] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4563.951394] Lustre: Skipped 2 previous similar messages [ 4564.385629] Lustre: server umount lustre-OST0000 complete [ 4569.060953] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4569.069699] Lustre: Skipped 2 previous similar messages [ 4569.074740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4569.082606] Lustre: Skipped 1 previous similar message [ 4574.065676] Lustre: server umount lustre-MDT0000 complete [ 4574.181398] LustreError: 160104:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4574.207811] LustreError: 160104:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4577.426384] LustreError: 160091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786816752 with bad export cookie 15103570338432274376 [ 4577.429453] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4577.444435] LustreError: 160091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4577.867249] Lustre: server umount lustre-MDT0001 complete [ 4589.403718] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 4592.490862] Key type lgssc unregistered [ 4592.819427] LNet: 164185:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4592.832632] LNetError: 164185:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4592.868566] LNet: Removed LNI 192.168.206.114@tcp [ 4593.812596] Key type .llcrypt unregistered [ 4593.814660] Key type ._llcrypt unregistered [ 4661.772632] Key type ._llcrypt registered [ 4661.774954] Key type .llcrypt registered [ 4661.860329] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 4662.564609] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4662.631921] alg: No test for adler32 (adler32-zlib) [ 4663.672250] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 4663.881094] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 4665.527177] Key type lgssc registered [ 4666.316042] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4672.646601] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4672.659314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4673.978024] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4677.577868] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4683.236064] LustreError: 166653:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4683.252576] LustreError: 166653:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4684.941953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4685.330840] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4689.497989] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4696.523238] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4701.836144] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4708.004722] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4708.317294] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4710.390945] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4713.706293] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4718.603379] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4720.713669] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4730.848783] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4730.856743] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4730.873882] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4733.922303] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4733.923545] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4733.927627] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4733.950859] Lustre: Skipped 1 previous similar message [ 4735.316497] Lustre: server umount lustre-OST0000 complete [ 4739.041764] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4739.052758] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4739.056412] Lustre: Skipped 1 previous similar message [ 4744.161649] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4744.170601] Lustre: Skipped 1 previous similar message [ 4744.775802] Lustre: server umount lustre-MDT0000 complete [ 4747.929855] LustreError: 166634:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786816922 with bad export cookie 15334175320798006278 [ 4747.932028] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4747.937319] LustreError: 166634:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4748.156413] Lustre: server umount lustre-MDT0001 complete [ 4756.810847] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 4758.880958] Key type lgssc unregistered [ 4759.093485] LNet: 170206:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4759.105236] LNetError: 170206:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4759.126594] LNet: Removed LNI 192.168.206.114@tcp [ 4759.760337] Key type .llcrypt unregistered [ 4759.764361] Key type ._llcrypt unregistered [ 4780.833180] Key type ._llcrypt registered [ 4780.835924] Key type .llcrypt registered [ 4780.935620] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 4781.819902] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4781.840147] alg: No test for adler32 (adler32-zlib) [ 4782.876664] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 4783.128894] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 4784.799141] Key type lgssc registered [ 4785.772724] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4800.954938] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 14:02:53 (1786816973) [ 4815.314865] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 4824.147493] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 4855.118789] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4855.135769] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4856.327490] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4856.354790] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4856.422853] Lustre: lustre-MDT0000: new disk, initializing [ 4856.479853] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4856.500114] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4860.471583] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4869.370939] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4869.483381] Lustre: 174447:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4869.564682] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4869.569294] Lustre: Skipped 1 previous similar message [ 4869.625655] Lustre: lustre-MDT0001: new disk, initializing [ 4869.698733] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4869.739408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4869.749054] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4873.012702] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4880.977578] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4886.190912] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4892.839889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4893.177490] Lustre: lustre-OST0000: new disk, initializing [ 4893.183093] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4893.189338] Lustre: 176412:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4893.270744] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4898.851995] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4898.859772] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4898.936721] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4899.167575] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4907.988682] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4919.271338] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4919.273290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4919.285603] Lustre: Skipped 1 previous similar message [ 4919.305824] Lustre: Skipped 1 previous similar message [ 4922.612385] Lustre: server umount lustre-OST0000 complete [ 4925.920543] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4925.925719] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4925.942501] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4929.505852] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4929.508621] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4931.876357] Lustre: server umount lustre-MDT0000 complete [ 4935.137576] LustreError: 174439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786817109 with bad export cookie 7519526065636380424 [ 4935.147565] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4935.553743] Lustre: server umount lustre-MDT0001 complete [ 4945.084120] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 4947.285111] Key type lgssc unregistered [ 4947.549526] LNet: 178318:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4947.566783] LNetError: 178318:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4947.583775] LNet: Removed LNI 192.168.206.114@tcp [ 4948.394276] Key type .llcrypt unregistered [ 4948.399865] Key type ._llcrypt unregistered [ 4967.768887] Key type ._llcrypt registered [ 4967.772768] Key type .llcrypt registered [ 4967.883709] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 4968.687205] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4968.706292] alg: No test for adler32 (adler32-zlib) [ 4969.704860] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 4969.901405] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 4971.527281] Key type lgssc registered [ 4972.435582] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4978.897986] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4978.918721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4980.318457] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4984.028190] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 4990.437586] LustreError: 179726:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4990.465825] LustreError: 179726:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4990.807041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4991.066476] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4994.554730] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 5000.945958] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5006.201651] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5012.601459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5012.895050] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5018.175134] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 5023.555693] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5025.197462] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5034.129137] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5034.147789] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5034.156556] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5034.165840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.180367] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5034.923670] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5034.943987] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5034.959516] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5034.966977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.984478] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5035.973841] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5035.995294] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5036.008627] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5036.014560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.028958] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5036.734557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.671070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.530594] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5038.541978] Lustre: Skipped 2 previous similar messages [ 5038.566939] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5038.585216] Lustre: Skipped 2 previous similar messages [ 5038.591267] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5038.598232] Lustre: Skipped 2 previous similar messages [ 5038.604487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.621243] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5038.632439] Lustre: Skipped 2 previous similar messages [ 5039.420944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.103461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.948865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.661574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.567706] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5042.574957] Lustre: Skipped 4 previous similar messages [ 5042.599401] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5042.611555] Lustre: Skipped 4 previous similar messages [ 5042.614256] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5042.626214] Lustre: Skipped 4 previous similar messages [ 5042.636644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.648867] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5042.661810] Lustre: Skipped 4 previous similar messages [ 5043.579583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.451227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.269897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.019154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.801278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.617471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.355324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.081168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.816821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.616133] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5050.621511] Lustre: Skipped 9 previous similar messages [ 5050.642506] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5050.652302] Lustre: Skipped 9 previous similar messages [ 5050.656109] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5050.661461] Lustre: Skipped 9 previous similar messages [ 5050.664934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.674554] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5050.680376] Lustre: Skipped 9 previous similar messages [ 5051.429885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.134276] LustreError: 179746:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5052.140959] LustreError: 179746:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5052.156425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.788773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.509400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.282590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.025710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.674243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.528110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.173169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.908530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.549046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.349872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.004624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.702531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.345924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.024823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.759469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.615073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.382914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.162216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.905588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.717634] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5066.730846] Lustre: Skipped 21 previous similar messages [ 5066.756666] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5066.767653] Lustre: Skipped 21 previous similar messages [ 5066.772505] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5066.777257] Lustre: Skipped 21 previous similar messages [ 5066.780748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.792112] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5066.799804] Lustre: Skipped 21 previous similar messages [ 5067.403596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.139310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.820415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.494873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.112383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.816363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.446724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.171951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.741223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.365272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.030180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.770689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.429468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.241696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.909224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.629817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.386709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.129479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.750490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.471713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.136514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.935713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.533749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.227106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.890640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.572461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.173950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.891101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.481520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.335518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.072550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.833972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.532831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.313449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.119359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.992714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.749146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.630467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.413860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.101048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.766701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.519502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.287817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.198918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.924263] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5098.929149] Lustre: Skipped 44 previous similar messages [ 5098.941396] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5098.950479] Lustre: Skipped 44 previous similar messages [ 5098.954023] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5098.958430] Lustre: Skipped 44 previous similar messages [ 5098.964435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.977916] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5098.986370] Lustre: Skipped 44 previous similar messages [ 5099.788570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.494896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.378256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.097759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.839873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.493586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.216841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.888715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.628158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.422891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.104070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.729141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.505572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.312620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.163720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.854443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.613127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.273523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.899907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.491818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.116211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.705840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.413926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.149554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.852459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.534622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.301971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.023697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.829815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.518328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.194322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.860931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.750716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.545063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.277950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.089899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.945992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.814605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.627687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.456832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.248156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.988779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.900566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.556718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.392446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.163455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.032326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.889352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.638287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.436093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.337655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.131618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.911879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.761523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.688477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.486109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.391384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.224970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.995484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.644942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.277847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.052918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.812876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.642614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.527589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.339589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.275485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.073968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.928449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.613573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.451358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.362707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.483265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.550319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.395172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.361492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.273489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.171512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.937644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.704147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.514779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.357316] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5163.366117] Lustre: Skipped 81 previous similar messages [ 5163.378565] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5163.393212] Lustre: Skipped 81 previous similar messages [ 5163.397660] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5163.402251] Lustre: Skipped 81 previous similar messages [ 5163.407447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.417985] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5163.421893] Lustre: Skipped 81 previous similar messages [ 5164.176409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.941718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.555122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.370104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.035251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.757418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.526778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.258369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.026781] LustreError: 179746:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5170.042157] LustreError: 179746:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5170.044426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.743808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.542892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.259819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.043326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.938302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.777574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.511056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.325865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.048910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.820920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.711309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.513256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.303468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.158957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.876859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.591261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.366609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.209077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.954254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.724579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.458110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.309147] LustreError: 179746:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5187.319113] LustreError: 179746:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5187.319773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.179340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.075646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.792466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.572884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.291679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.095555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.825839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.671940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.448243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.447124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.183665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.017921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.755400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.528815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.353182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.253182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.108256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.961565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.680315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.450918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.070266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.994475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.052262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.954070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.678113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.617129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.415418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.299429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.286668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.307367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.428082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.357608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.651458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.655601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.842656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.885711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.966767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.930062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.877515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.724441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.538740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.608863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.517783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.647193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.631528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.531880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.354956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.275635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.316701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.383719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.486624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.392261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.326905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.198473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.130150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.901581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.132850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.082490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.015518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.892684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.039514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.317499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.268770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.422746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.995651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.673819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.990349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.784520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.916117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.942988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.050533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.229695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.315251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.706359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.968639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.106479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.038087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.982286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.689796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.520396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.488624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.284144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.450456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.296539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.157604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.050387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.963103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.781418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.717472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.507628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.404244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.171700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.130772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.956650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.946970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.708758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.400731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.323130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.316812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.270652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.974356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.690438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.562667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.556792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.601737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.551125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.348976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.187579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.987843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.867570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.801965] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5291.808754] Lustre: Skipped 141 previous similar messages [ 5291.826934] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5291.833697] Lustre: Skipped 141 previous similar messages [ 5291.839577] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5291.847510] Lustre: Skipped 141 previous similar messages [ 5291.849707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.861194] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5291.869042] Lustre: Skipped 141 previous similar messages [ 5292.828418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.733980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.620815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.560512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.438738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.325866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.375491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.217237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.097192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.115837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.938839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.784872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.534244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.280020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.015409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.707792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.496460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.146108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.049746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.966674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.736336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.591034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.317062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.196827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.959594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.624743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.403654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.195822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.020775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.925164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.868822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.745110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.744655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.615907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.419180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.285646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.049604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.744736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.464528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.137986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.710728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.282091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.834502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.450673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.056181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.792924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.437984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.135458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.869181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.615471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.397040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.305770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.152798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.037249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.080462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.840082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.512974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.350177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.397553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.384173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.039363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.865624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.045766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.380450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.329252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.681961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.574294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.577873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.541681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.505244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.370832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.233892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.237549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.282899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.225856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.232888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.121978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.122481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.302232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.255526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.338477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.123799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.205824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.255352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.197057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.167916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.145471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.308873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.327437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.391429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.630936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.532694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.588481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.365584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.338505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.348083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.293294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.073647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.052268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.212232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.161541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.138256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.018009] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.788762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.605164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.422730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.319333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.180099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.066476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.909253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.863077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.725325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.587084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.519567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.352472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.155200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.925715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.682259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.648732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.387081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.217546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.854409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.632154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.381573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.196356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.955820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.707809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.453129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.247486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.075267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.890755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.804684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.830065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.711074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.494091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.566850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.320365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.240327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.041499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.856379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.587697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.532256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.444673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.463824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.408627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.440214] LustreError: 179746:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5419.448169] LustreError: 179746:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5419.466685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.420303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.386818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.454712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.328128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.235789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.979907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.648981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.555818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.400361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.283239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.386038] LustreError: 179746:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5429.399845] LustreError: 179746:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5429.420425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.589806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.478985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.515247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.277123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.464790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.618472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.857395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.897974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.918973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.865950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.712077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.609944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.321774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.137654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.993310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.703283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.553387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.314950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.187148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.938712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.753959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.755970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.973450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.384620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.552587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.521756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.350575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.120578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.157811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.158497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.186080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.256615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.482651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.561596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.577508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.290504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.973029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.901256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.730260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.891185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.916484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.786211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.573439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.474795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.259315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.145985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.060733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.218267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.263988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.161757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.124972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.107522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.826869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.721635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.465095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.410826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.320740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.130589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.920857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.715881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.480347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.526760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.534734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.391703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.463733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.285936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.352509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.084837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.962327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.695398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.530586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.487631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.543849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.561537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.519400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.280882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.001418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.788269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.564630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.379095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.268345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.133276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.850955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.753984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.545427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.602329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.438332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.190507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.988435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.883498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.728884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.722336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.543875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.416192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.184892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.129301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.961537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.854081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.794945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.761607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.838763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.784462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.652430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.268109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.310815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.096722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.127256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.762648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.651937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.452330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.340313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.153255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.052443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.991671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.994408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.817703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.653305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.674648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.756069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.887863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.074033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.177983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.184477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.165245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.149595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.330680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.396965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.746846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.873391] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5548.876561] Lustre: Skipped 285 previous similar messages [ 5548.909539] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5548.921990] Lustre: Skipped 285 previous similar messages [ 5548.927804] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5548.933404] Lustre: Skipped 285 previous similar messages [ 5548.943146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.956581] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5548.975207] Lustre: Skipped 285 previous similar messages [ 5549.720560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.694442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.838639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.370852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.871035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.109290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.105412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.167538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.380840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.593294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.923988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.196216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.209325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.366970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.491905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.509977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.806297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.204650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.680021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.958665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.833415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.827732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.748766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.780683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.609427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.440315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.928438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.882966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.036209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.855383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.903502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.824553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.816629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.668703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.590349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.291573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.228742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.139944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.146209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.258391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.035681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.950514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.975470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.798736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.672220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.625904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.487519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.314805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.107625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.930783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.735584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.508046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.343486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.318218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.186832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.144429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.972170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.067544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.965456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.919341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.873963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.687211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.537933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.439314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.303939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.270747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.287699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.039495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.946750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.879369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.816813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.779448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.678763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.691538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.827090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.704062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.566725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.430401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.321405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.314690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.168853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.077427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.109803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.060063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.006917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.862357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.908417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.798175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.858454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.633303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.472599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.345241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.216433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.930720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.777820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.452689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.254822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.981984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.705916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.513439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.270470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.056502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.889332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.821584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.594920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.256982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.130797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.827389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.678429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.506498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.232403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.969911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.783992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.584483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.426256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.202804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.973436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.827834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.722886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.341504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.143783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.935102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.854256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.795468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.666974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.490796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.269378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.163996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.936581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.778657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.600343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.429767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.409591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.480442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.471090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.376119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.130063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.944417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.867459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.678603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.640302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.625756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.598234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.507787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.423305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.193538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.968815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.904900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.872469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.874856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.701426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.525656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.376573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.293330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.387627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.357453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.290467] LustreError: 179746:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5694.296954] LustreError: 179746:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5694.312787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.179182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.095554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.905502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.961856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.986045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.913409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.670634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.605228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.459276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.385349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.169303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.162132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.155447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.266765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.166968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.163731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.992221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.973342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.793657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.814870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.730883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.791267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.813704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.839365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.672330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.458588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.145598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.068612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.860255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.916297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.791197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.758207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.697731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.722498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.693797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.794919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.586484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.554501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.382266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.261307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.201452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.111673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.914304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.847278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.816920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.688494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.500550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.593681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.388824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.011319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.696206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.605703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.452325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.471457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.439764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.367959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.116637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.952696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.789914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.652523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.767720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.604670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.486442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.345185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.190633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.441690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.222582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.025235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.085173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.865338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.773742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.670245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.445531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.519504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.423719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.257598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.035783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.947644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.773074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.529910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.273919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.049738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.692488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.503685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.246000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.054629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.767909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.651768] LustreError: 179746:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5772.660646] LustreError: 179746:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5772.665886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.453076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.176166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.893161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.717906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.556631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.463097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.272416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.302147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.204403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.093816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.945821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.874536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.571193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.420517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.242518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.072564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.883150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.858159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.569467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.352741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.060460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.788774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.607826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.335188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.029191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.810362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.628790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.501252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.374029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.163618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.033431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.709460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.456518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.242275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.052536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.935735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.724067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.473423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.224647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.991426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.848592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.740045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.386961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.029341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.623531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.482408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.336842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.134852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.921473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.986248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.926371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.923879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.177606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.385644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.634967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.676707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.623667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.565801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.245726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.457332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.489733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.315171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.240884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.272214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.531831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.359430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.182477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.181717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.963221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.972187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.017084] LustreError: 179746:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5834.031492] LustreError: 179746:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5834.043307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.205444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.902964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.670451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.424132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.662049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.872743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.411513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.616870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.767849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.921108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.995445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.105218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.104547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.365904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.420297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.634954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.784813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.964077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.111029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.948072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.910991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.753573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.625496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.302582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.203111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.023080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.901479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.810755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.819488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.862122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.824396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.657620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.682442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.593132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.565095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.378921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.178959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.998918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.011484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.801295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.767685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.764228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.483938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.363334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.416644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.309126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.319570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.277177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.110696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.823707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.759898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.618143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.685192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.418932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.152417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.118856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.250557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.056099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.889928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.606706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.418898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.247545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.184609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.920318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.695865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.857985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.936952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.072383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.045837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.186800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.191904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.148668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.998506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.988748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.877966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.853787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.709356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.675057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.690646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.691462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.868341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.796306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.793677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.388215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.247460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.941530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.225479] LustreError: 181550:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6286.245805] LustreError: 179746:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6940.642179] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6940.659346] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6940.687525] Lustre: Skipped 402 previous similar messages [ 6940.707694] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6943.566754] Lustre: server umount lustre-OST0000 complete [ 6951.396520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6951.404738] Lustre: Skipped 2 previous similar messages [ 6953.588444] Lustre: server umount lustre-MDT0000 complete [ 6956.514033] LustreError: 180810:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6956.546719] LustreError: 180810:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6957.080415] LustreError: 197251:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786819131 with bad export cookie 3988193416261296099 [ 6957.085762] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6957.090216] LustreError: 197251:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6957.378716] Lustre: server umount lustre-MDT0001 complete [ 6968.176153] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 6970.903252] Key type lgssc unregistered [ 6971.145740] LNet: 234966:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6971.159105] LNetError: 234966:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6971.179451] LNet: Removed LNI 192.168.206.114@tcp [ 6972.025814] Key type .llcrypt unregistered [ 6972.028538] Key type ._llcrypt unregistered [ 6994.079813] Key type ._llcrypt registered [ 6994.082640] Key type .llcrypt registered [ 6994.185842] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 6994.871328] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6994.888968] alg: No test for adler32 (adler32-zlib) [ 6995.948362] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 6996.154764] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 6997.805369] Key type lgssc registered [ 6998.642458] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7005.939914] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7005.954447] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7007.371124] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7011.781452] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7017.957211] LustreError: 236385:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7017.970946] LustreError: 236385:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7019.473816] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7019.820181] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7023.721097] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7030.201875] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7034.942583] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7040.400297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7040.645262] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7045.173969] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7051.702680] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7052.978087] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7057.055923] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7059.069658] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7059.085118] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7059.093397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7059.100413] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7059.103092] LustreError: 236404:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7063.526875] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7063.530792] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7063.539086] Lustre: Skipped 1 previous similar message [ 7063.547443] Lustre: Skipped 1 previous similar message [ 7064.991524] Lustre: lustre-OST0000: Not available for connect from 192.168.206.14@tcp (stopping) [ 7068.641357] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7068.652222] Lustre: Skipped 1 previous similar message [ 7073.761657] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7073.774796] Lustre: Skipped 2 previous similar messages [ 7075.295180] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7075.433381] Lustre: server umount lustre-OST0000 complete [ 7079.111129] LustreError: 236404:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7107.868471] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7108.163961] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7113.132758] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7118.783417] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7128.678394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7128.694518] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7128.705247] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7128.711958] Lustre: Skipped 1 previous similar message [ 7132.054605] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 11 sec [ 7137.076947] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9b7d45346000.ost_server_uuid 50 [ 7138.720912] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9b7d45346000.ost_server_uuid in FULL state after 0 sec [ 7144.418526] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7144.434127] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7144.435413] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7144.454013] Lustre: Skipped 2 previous similar messages [ 7145.762524] Lustre: server umount lustre-OST0000 complete [ 7153.120712] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7153.128751] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7153.153625] Lustre: Skipped 1 previous similar message [ 7153.162475] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7156.724785] Lustre: server umount lustre-MDT0000 complete [ 7159.777750] LustreError: 236385:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7159.796519] LustreError: 236385:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7160.194494] LustreError: 239116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786819335 with bad export cookie 8627657738713036302 [ 7160.197066] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7160.203590] LustreError: 239116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7160.457496] Lustre: server umount lustre-MDT0001 complete [ 7169.213434] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 7171.240973] Key type lgssc unregistered [ 7171.439119] LNet: 241622:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7171.444709] LNetError: 241622:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7171.458924] LNet: Removed LNI 192.168.206.114@tcp [ 7172.111550] Key type .llcrypt unregistered [ 7172.113932] Key type ._llcrypt unregistered [ 7197.362738] Key type ._llcrypt registered [ 7197.365322] Key type .llcrypt registered [ 7197.456871] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 7198.214314] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7198.247339] alg: No test for adler32 (adler32-zlib) [ 7199.228926] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 7199.414951] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 7201.071950] Key type lgssc registered [ 7202.083693] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7209.262278] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7209.276304] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7210.652803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7214.779147] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7220.194134] LustreError: 243344:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7220.209651] LustreError: 243344:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7223.460488] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7223.866329] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7228.161654] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7231.527974] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7238.483279] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7238.865824] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7245.231868] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7253.321909] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7253.503241] Lustre: lustre-OST0001: new disk, initializing [ 7253.505921] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7253.510925] Lustre: 245651:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7253.593887] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7255.689956] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7259.863220] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7262.283290] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7262.293477] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7262.392227] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7270.766414] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7278.060126] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7293.416586] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7293.418481] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7293.431733] Lustre: Skipped 2 previous similar messages [ 7293.434870] Lustre: Skipped 1 previous similar message [ 7297.722615] Lustre: server umount lustre-MDT0000 complete [ 7298.532540] LustreError: 243345:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7298.546575] LustreError: 243345:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7300.852331] LustreError: 243325:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786819475 with bad export cookie 14059706414234246207 [ 7300.855251] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7300.862195] LustreError: 243325:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7301.300836] Lustre: server umount lustre-MDT0001 complete [ 7316.108715] Lustre: server umount lustre-OST0000 complete [ 7319.818635] Lustre: 242606:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786819478/real 1786819478] req@ffff9705d03b9c00 x1873615912009216/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786819494 ref 1 fl Rpc:EXNQr/200/ffffffff rc -5/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7330.185847] Lustre: server umount lustre-OST0001 complete [ 7351.706775] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 7360.696066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7361.010034] LustreError: 248765:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7361.085718] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7364.829801] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7366.124336] LustreError: 248766:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7370.210898] LustreError: 248765:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7372.419241] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7372.733874] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7376.949537] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7379.902408] Lustre: 249906:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7386.696680] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7393.351398] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7397.294091] LustreError: 250260:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7401.278732] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7401.557931] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7401.564289] Lustre: Skipped 1 previous similar message [ 7405.720189] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7407.538951] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7414.875232] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7418.863684] Lustre: 251780:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7429.089442] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7429.100158] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7429.118368] Lustre: Skipped 1 previous similar message [ 7429.127973] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7429.132644] Lustre: Skipped 2 previous similar messages [ 7431.654499] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7431.655893] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7431.660632] Lustre: Skipped 1 previous similar message [ 7431.678883] Lustre: Skipped 2 previous similar messages [ 7434.010103] Lustre: server umount mylustre-MDT0000 complete [ 7436.775398] LustreError: 249497:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7436.803574] LustreError: 249497:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7437.587399] LustreError: 249508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786819612 with bad export cookie 14059706414234249014 [ 7437.589838] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7437.596922] LustreError: 249508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7438.075758] Lustre: server umount mylustre-MDT0001 complete [ 7452.248643] Lustre: server umount mylustre-OST0000 complete [ 7465.866978] Lustre: server umount mylustre-OST0001 complete [ 7488.122319] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 7497.570461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7498.111394] LustreError: 254090:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7498.190035] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7502.389837] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7511.438792] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7517.281422] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7520.247666] Lustre: 255233:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7526.437447] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7531.796423] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7539.775187] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7543.095227] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7545.757803] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7552.947996] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7555.978719] Lustre: 257104:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7563.232633] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7563.243803] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7563.278166] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7563.290218] Lustre: Skipped 2 previous similar messages [ 7564.349917] Lustre: server umount tfs-MDT0000 complete [ 7568.119632] LustreError: 254065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786819742 with bad export cookie 14059706414234250680 [ 7568.130362] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7568.134298] LustreError: 254065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7568.539570] Lustre: server umount tfs-MDT0001 complete [ 7581.552928] Lustre: server umount tfs-OST0000 complete [ 7584.482451] Lustre: 242606:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786819743/real 1786819743] req@ffff9705e6839500 x1873615912113152/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786819759 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7584.498991] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7584.508836] Lustre: Skipped 3 previous similar messages [ 7585.209616] Lustre: server umount tfs-OST0001 complete [ 7604.706837] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 7613.955619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7614.280596] LustreError: 259352:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7614.298606] LustreError: 259352:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7614.367329] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7614.377387] Lustre: Skipped 3 previous similar messages [ 7618.898593] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7627.452898] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7631.741921] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7635.111284] Lustre: 260495:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7641.838286] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7648.621995] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7657.443223] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7659.711712] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7663.932246] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7671.039138] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7674.628416] Lustre: 262367:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7681.541669] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 14:50:54 (1786819854) [ 7699.424481] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7699.429249] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7699.437176] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7699.442520] Lustre: Skipped 3 previous similar messages [ 7703.837079] Lustre: server umount lustre-MDT0000 complete [ 7707.273128] LustreError: 259328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786819882 with bad export cookie 14059706414234252122 [ 7707.277886] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7707.282785] LustreError: 259328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7711.224461] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7711.236782] Lustre: Skipped 5 previous similar messages [ 7713.848400] Lustre: server umount lustre-MDT0001 complete [ 7724.342863] Lustre: server umount lustre-OST0000 complete [ 7733.904625] Lustre: server umount lustre-OST0001 complete [ 7740.598588] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 7750.733880] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 7785.832639] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7785.832639] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7785.832639] [ 7785.859560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7786.082303] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7786.115400] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7786.289279] Lustre: lustre-MDT0000: new disk, initializing [ 7786.378473] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7786.382814] Lustre: Skipped 3 previous similar messages [ 7786.406759] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7791.326505] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7800.773033] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7800.773033] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7800.773033] [ 7800.798095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7800.900946] Lustre: 265554:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7800.947835] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7800.954213] Lustre: Skipped 1 previous similar message [ 7801.025427] Lustre: lustre-MDT0001: new disk, initializing [ 7801.121974] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7801.128718] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7804.914910] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7814.833263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7815.069729] Lustre: lustre-OST0000: new disk, initializing [ 7815.080756] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7815.093474] Lustre: 267134:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7816.788459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7816.800551] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7816.907862] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7820.424937] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7828.899659] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7829.048880] Lustre: lustre-OST0001: new disk, initializing [ 7829.052573] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7829.058085] Lustre: 268008:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7830.398047] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7830.409623] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7830.505473] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7834.943899] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 7845.605074] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 14:53:38 (1786820018) [ 7858.172280] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 14:53:51 (1786820031) [ 7873.506345] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7873.506896] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7873.509302] LustreError: Skipped 1 previous similar message [ 7873.517513] Lustre: Skipped 5 previous similar messages [ 7873.518280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7873.542748] Lustre: Skipped 3 previous similar messages [ 7887.327164] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7887.450062] Lustre: server umount lustre-OST0000 complete [ 7891.943822] LustreError: 267255:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7891.964657] LustreError: 267255:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 7897.396047] Lustre: server umount lustre-MDT0000 complete [ 7900.846429] LustreError: 267133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786820075 with bad export cookie 14059706414234253445 [ 7900.867407] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7900.867721] LustreError: 267133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7907.481172] Lustre: server umount lustre-MDT0001 complete [ 7920.654197] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 7998.947475] Lustre: server umount lustre-OST0001 complete [ 8007.385471] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 8022.731139] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 8062.140298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8062.388368] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8062.444192] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8062.544901] Lustre: lustre-MDT0000: new disk, initializing [ 8062.623984] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8062.629243] Lustre: Skipped 3 previous similar messages [ 8062.643065] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8068.187154] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8078.534676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8078.673826] Lustre: 272811:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8078.750204] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8078.755430] Lustre: Skipped 1 previous similar message [ 8078.824509] Lustre: lustre-MDT0001: new disk, initializing [ 8079.017125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8079.034561] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8083.443398] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8092.118694] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8097.404682] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8103.144310] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8103.589716] Lustre: 274780:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8105.207428] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8105.326055] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8109.256799] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8118.772570] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8131.044969] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8131.058912] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8131.064627] Lustre: Skipped 5 previous similar messages [ 8131.075453] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8131.081368] Lustre: Skipped 15 previous similar messages [ 8132.370249] Lustre: server umount lustre-OST0000 complete [ 8140.259050] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8142.600920] Lustre: server umount lustre-MDT0000 complete [ 8145.878737] LustreError: 273607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786820320 with bad export cookie 14059706414234254922 [ 8145.880719] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8145.889526] LustreError: 273607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8146.260295] Lustre: server umount lustre-MDT0001 complete [ 8155.784480] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 8158.125654] Key type lgssc unregistered [ 8158.373862] LNet: 276544:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8158.379703] LNetError: 276544:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8158.392277] LNet: Removed LNI 192.168.206.114@tcp [ 8159.087503] Key type .llcrypt unregistered [ 8159.089910] Key type ._llcrypt unregistered [ 8181.368627] Key type ._llcrypt registered [ 8181.373221] Key type .llcrypt registered [ 8181.467885] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 8182.414551] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8182.434968] alg: No test for adler32 (adler32-zlib) [ 8183.421705] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 8183.603605] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 8185.271151] Key type lgssc registered [ 8186.355668] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8192.935076] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8192.962807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8194.294670] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8197.923604] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8204.770784] LustreError: 277986:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8204.792508] LustreError: 277986:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8205.236116] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8205.571474] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8209.132514] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8216.815754] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8221.933101] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8227.684388] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8227.964019] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8233.880191] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8239.231610] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8240.263970] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8240.969726] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8244.707443] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8244.708672] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8244.713750] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8244.716907] Lustre: Skipped 1 previous similar message [ 8248.511538] Lustre: server umount lustre-OST0000 complete [ 8251.872078] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8251.876292] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8251.890165] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8251.897027] Lustre: Skipped 1 previous similar message [ 8254.948887] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8254.952445] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8257.969397] Lustre: server umount lustre-MDT0000 complete [ 8260.071724] LustreError: 277986:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8260.099854] LustreError: 277986:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8261.698059] LustreError: 277966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786820436 with bad export cookie 14883974878648589621 [ 8261.701135] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8261.713247] LustreError: 277966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8262.084675] Lustre: server umount lustre-MDT0001 complete [ 8272.439547] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 8274.726711] Key type lgssc unregistered [ 8274.941655] LNet: 281572:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8274.947987] LNetError: 281572:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8274.971049] LNet: Removed LNI 192.168.206.114@tcp [ 8275.875832] Key type .llcrypt unregistered [ 8275.877902] Key type ._llcrypt unregistered [ 8292.007487] Key type ._llcrypt registered [ 8292.009841] Key type .llcrypt registered [ 8292.103979] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 8292.994055] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8293.043846] alg: No test for adler32 (adler32-zlib) [ 8294.015794] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 8294.319353] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 8296.039151] Key type lgssc registered [ 8297.187862] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8305.826422] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8305.848854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8321.375455] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8326.495584] LustreError: 282777:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.114@tcp: failed processing log, type 4: rc = -110 [ 8353.183457] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8358.210332] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8364.883965] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8366.495389] Lustre: server umount lustre-OST0000 complete [ 8380.024100] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 8382.184924] Key type lgssc unregistered [ 8382.510381] LNet: 284411:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8382.525464] LNetError: 284411:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8382.540384] LNet: Removed LNI 192.168.206.114@tcp [ 8383.234516] Key type .llcrypt unregistered [ 8383.236571] Key type ._llcrypt unregistered [ 8421.865557] Key type ._llcrypt registered [ 8421.867776] Key type .llcrypt registered [ 8421.997468] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 8423.448202] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8423.507537] alg: No test for adler32 (adler32-zlib) [ 8424.612959] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 8424.847421] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 8426.479273] Key type lgssc registered [ 8427.831902] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8441.395384] loop0: detected capacity change from 0 to 204800000 [ 8441.638824] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8441.654777] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8442.027807] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8443.005699] loop0: detected capacity change from 0 to 204800000 [ 8443.204624] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8443.227562] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8443.658210] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8444.665143] loop0: detected capacity change from 0 to 204800000 [ 8444.805540] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8444.822205] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8444.991055] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8446.028143] loop0: detected capacity change from 0 to 204800000 [ 8446.211902] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8447.344744] loop0: detected capacity change from 0 to 204800000 [ 8447.386819] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8450.984715] loop0: detected capacity change from 0 to 204800000 [ 8451.039106] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8454.931493] loop0: detected capacity change from 0 to 204800000 [ 8454.959054] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8458.577599] loop0: detected capacity change from 0 to 204800000 [ 8458.597533] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8462.103384] loop0: detected capacity change from 0 to 204800000 [ 8462.173849] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8462.218610] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8462.284384] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8462.292440] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8462.303382] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8462.315563] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8462.327601] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8462.340936] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8462.350459] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8462.363143] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8462.381684] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8462.395476] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8462.406799] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8462.417274] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8462.431829] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8462.449122] Lustre: 287823:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8467.736652] Lustre: server umount MGS complete [ 8468.539733] loop0: detected capacity change from 0 to 204800000 [ 8468.647693] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8468.955505] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8469.008727] Lustre: lustre-MDT0000: Aborting client recovery [ 8469.828283] loop1: detected capacity change from 0 to 204800000 [ 8470.061646] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8470.158166] Lustre: 288255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8470.175516] Lustre: 288255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8470.195304] Lustre: 288255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8470.206102] Lustre: 288255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8470.221648] Lustre: 288255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8470.237620] Lustre: 288255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8470.256338] Lustre: 288255:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8470.286100] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8470.531197] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8470.572132] Lustre: lustre-MDT0001: Aborting client recovery [ 8471.341694] loop2: detected capacity change from 0 to 204800000 [ 8471.461849] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8471.550975] Lustre: 288334:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8471.556957] Lustre: 288334:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8471.571920] Lustre: 288334:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8471.581830] Lustre: 288334:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8471.590232] Lustre: 288334:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8471.597680] Lustre: 288334:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8471.606878] Lustre: 288334:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8471.703533] Lustre: Found index 0 for lustre-OST0000, updating log [ 8471.727936] Lustre: 288352:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8471.783839] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8471.800488] Lustre: lustre-OST0000: Aborting recovery [ 8472.639341] loop3: detected capacity change from 0 to 204800000 [ 8472.745206] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8472.822563] Lustre: 288417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8472.833560] Lustre: 288417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8472.847964] Lustre: 288417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8472.861976] Lustre: 288417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8472.876646] Lustre: 288417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8472.885366] Lustre: 288417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8472.897952] Lustre: 288417:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8472.913572] Lustre: Found index 1 for lustre-OST0001, updating log [ 8472.941620] Lustre: 288422:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8472.995333] Lustre: lustre-OST0001: Aborting recovery [ 8478.188313] LustreError: 288199:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8479.226218] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8479.241764] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8479.290817] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8479.296371] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8480.237683] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8480.267576] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8480.911588] LustreError: 288166:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 5391us: rc = -2 [ 8481.424931] LustreError: 288432:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1316us: rc = -2 [ 8481.448038] LustreError: 288432:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 15 previous similar messages [ 8495.585802] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8495.589359] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8495.595504] Lustre: Skipped 2 previous similar messages [ 8495.599173] Lustre: Skipped 1 previous similar message [ 8500.410195] Lustre: server umount lustre-MDT0000 complete [ 8500.705250] LustreError: 288165:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8500.718317] LustreError: 288165:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 8501.086912] LustreError: 288148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786820675 with bad export cookie 6279804461797881700 [ 8501.095955] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8501.099670] LustreError: 288148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8505.826322] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8505.827202] LustreError: 288432:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8505.831179] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8505.831187] Lustre: Skipped 2 previous similar messages [ 8505.840176] Lustre: Skipped 2 previous similar messages [ 8505.876402] LustreError: 288432:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8506.848698] LustreError: 288432:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8506.851115] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8506.872697] Lustre: Skipped 2 previous similar messages [ 8507.603105] Lustre: server umount lustre-MDT0001 complete [ 8514.533214] Lustre: server umount lustre-OST0000 complete [ 8521.366448] Lustre: server umount lustre-OST0001 complete [ 8528.303486] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 15:05:01 (1786820701) [ 8553.537660] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 8561.321813] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 8596.173068] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8596.342706] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8596.379258] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8596.431312] Lustre: lustre-MDT0000: new disk, initializing [ 8596.494734] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8596.498205] Lustre: Skipped 1 previous similar message [ 8596.505379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8599.876104] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8609.230248] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8609.371582] Lustre: 292484:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8609.414785] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8609.419707] Lustre: Skipped 1 previous similar message [ 8609.479361] Lustre: lustre-MDT0001: new disk, initializing [ 8609.564839] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8609.601759] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8609.611271] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8613.184849] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8621.635878] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8626.130868] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8631.660384] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8631.864571] Lustre: lustre-OST0000: new disk, initializing [ 8631.870670] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8631.875385] Lustre: 294452:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8631.945248] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8637.917523] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8641.581019] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8641.588912] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8641.703191] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8646.734472] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8651.617697] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8664.585875] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8669.984458] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8723.937565] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8723.945092] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8723.958187] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8723.968435] Lustre: Skipped 1 previous similar message [ 8727.511707] Lustre: server umount lustre-OST0000 complete [ 8732.643277] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8732.660790] LustreError: Skipped 1 previous similar message [ 8732.664458] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8732.674370] Lustre: Skipped 1 previous similar message [ 8732.677977] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8737.837618] Lustre: server umount lustre-MDT0000 complete [ 8739.299953] LustreError: 292497:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8739.317254] LustreError: 292497:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8741.203178] LustreError: 292478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786820916 with bad export cookie 6279804461797884402 [ 8741.209949] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8741.213607] LustreError: 292478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8741.553545] Lustre: server umount lustre-MDT0001 complete [ 8748.995808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8752.888866] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8756.858829] Lustre: server umount MGS complete [ 8765.515358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8765.914034] LustreError: 298387:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8765.938682] LustreError: 298400:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8765.995552] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8769.695984] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8776.161486] LustreError: 298388:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8776.167972] LustreError: 298388:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8777.155492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8781.024292] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8788.768255] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8794.251679] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8800.394957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8800.886640] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8800.893460] Lustre: Skipped 1 previous similar message [ 8806.721913] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8814.396928] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8841.375804] Lustre: server umount lustre-OST0000 complete [ 8847.329548] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8847.339615] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8847.347803] Lustre: Skipped 2 previous similar messages [ 8847.359885] Lustre: Skipped 3 previous similar messages [ 8854.587401] Lustre: server umount lustre-MDT0000 complete [ 8857.571251] LustreError: 298388:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8857.601700] LustreError: 298388:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8857.856303] LustreError: 301111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786821032 with bad export cookie 6279804461797886243 [ 8857.862488] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8857.868677] LustreError: 301111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8867.472237] Lustre: server umount lustre-MDT0001 complete [ 8876.772639] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 8879.208385] Key type lgssc unregistered [ 8879.532720] LNet: 302336:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8879.549664] LNetError: 302336:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8879.574669] LNet: Removed LNI 192.168.206.114@tcp [ 8880.439749] Key type .llcrypt unregistered [ 8880.441905] Key type ._llcrypt unregistered [ 8915.638842] Key type ._llcrypt registered [ 8915.640967] Key type .llcrypt registered [ 8915.752104] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 8930.615388] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 8931.425396] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8931.464470] alg: No test for adler32 (adler32-zlib) [ 8932.519605] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 8932.728854] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 8934.367170] Key type lgssc registered [ 8935.270678] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8968.285260] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8968.322797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8969.540888] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8969.571274] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8969.653868] Lustre: lustre-MDT0000: new disk, initializing [ 8969.716577] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8969.746971] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8974.123595] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8983.010875] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8983.111340] Lustre: 306296:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8983.183312] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8983.186248] Lustre: Skipped 1 previous similar message [ 8983.242432] Lustre: lustre-MDT0001: new disk, initializing [ 8983.319100] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8983.344604] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8983.352290] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8987.137753] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 8995.216613] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8999.814655] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9005.241474] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9005.520847] Lustre: lustre-OST0000: new disk, initializing [ 9005.524813] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9005.532847] Lustre: 308265:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9005.603980] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9011.363355] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 9014.810138] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9014.823838] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9014.910530] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9019.386385] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9026.257966] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9039.331697] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9043.575971] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9049.107529] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9094.633841] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9094.649824] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9094.664635] Lustre: Skipped 1 previous similar message [ 9094.677412] Lustre: Skipped 1 previous similar message [ 9099.391680] Lustre: server umount lustre-OST0000 complete [ 9104.865279] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9104.879401] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9104.886167] Lustre: Skipped 1 previous similar message [ 9109.197389] Lustre: server umount lustre-MDT0000 complete [ 9112.971982] LustreError: 308264:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786821287 with bad export cookie 13526169390498631818 [ 9112.975512] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9112.990041] LustreError: 308264:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9113.330913] Lustre: server umount lustre-MDT0001 complete [ 9123.596393] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9130.103824] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 9135.655456] Lustre: server umount MGS complete [ 9145.033363] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9145.353972] LustreError: 312134:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9145.381013] LustreError: 312134:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9145.393507] LustreError: 312147:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9145.472889] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9148.992086] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 9150.953230] LustreError: 312135:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9156.069488] LustreError: 312134:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9157.604740] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9158.000868] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9161.824895] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 9168.376775] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9173.467366] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9179.952421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9180.383787] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9186.270648] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 9193.471855] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9221.599877] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9221.606625] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9221.615649] Lustre: Skipped 1 previous similar message [ 9221.622521] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9221.985185] Lustre: server umount lustre-OST0000 complete [ 9229.799785] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9229.805640] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9229.809248] Lustre: Skipped 1 previous similar message [ 9229.809599] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9231.616861] Lustre: server umount lustre-MDT0000 complete [ 9234.537293] LustreError: 312115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786821409 with bad export cookie 13526169390498633743 [ 9234.539837] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9234.544741] LustreError: 312115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9245.314790] Lustre: server umount lustre-MDT0001 complete [ 9255.073804] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [ 9257.869841] Key type lgssc unregistered [ 9258.160888] LNet: 316080:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9258.166774] LNetError: 316080:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9258.180741] LNet: Removed LNI 192.168.206.114@tcp [ 9258.760565] Key type .llcrypt unregistered [ 9258.763720] Key type ._llcrypt unregistered [ 9277.000449] Key type ._llcrypt registered [ 9277.002961] Key type .llcrypt registered [ 9277.083674] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [ 9287.433259] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 9288.004272] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9288.042183] alg: No test for adler32 (adler32-zlib) [ 9288.970140] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [ 9289.115840] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [ 9290.719263] Key type lgssc registered [ 9291.703967] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9315.269713] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [ 9330.064617] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9330.117556] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9331.314876] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9331.355394] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9331.469814] Lustre: lustre-MDT0000: new disk, initializing [ 9331.568282] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9331.598751] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9335.662973] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [ 9345.189883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9345.470738] Lustre: lustre-OST0000: new disk, initializing [ 9345.473986] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9345.478158] Lustre: Skipped 1 previous similar message [ 9345.494270] Lustre: 320111:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9345.593771] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9348.454294] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9348.461539] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9348.495503] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9350.359363] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10162.660738] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10162.670862] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10167.777505] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10169.105565] Lustre: server umount lustre-MDT0000 complete [10173.162424] LustreError: 319299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786822347 with bad export cookie 10591621970185886501 [10173.173352] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10173.410921] Lustre: server umount lustre-OST0000 complete [10193.956454] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [10202.343058] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [10245.673690] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10245.912466] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10245.947318] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10246.038940] Lustre: lustre-MDT0000: new disk, initializing [10246.130809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10246.154711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10250.502253] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10259.580824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10259.705312] Lustre: 324440:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10259.763131] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10259.767169] Lustre: Skipped 1 previous similar message [10259.843632] Lustre: lustre-MDT0001: new disk, initializing [10259.934254] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10259.980197] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10259.994312] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10264.451968] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10272.150562] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10276.761855] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10282.592169] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10282.939620] Lustre: lustre-OST0000: new disk, initializing [10282.947478] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10282.965396] Lustre: 326407:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10283.075318] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10288.622763] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10290.241199] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10290.264579] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10290.320643] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10297.268872] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10302.067232] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10302.320138] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10306.788569] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10307.017456] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10311.137681] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10311.150866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10311.154715] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10311.158409] Lustre: Skipped 1 previous similar message [10314.879748] Lustre: server umount lustre-OST0000 complete [10321.377870] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10321.382569] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10321.382822] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10321.392433] Lustre: Skipped 2 previous similar messages [10324.856202] Lustre: server umount lustre-MDT0000 complete [10327.798585] LustreError: 324434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786822502 with bad export cookie 10591621970188827320 [10327.801415] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10327.808102] LustreError: 324434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10328.067263] Lustre: server umount lustre-MDT0001 complete [10331.163972] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10332.878501] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 15:35:05 (1786822505) [10339.235880] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10339.529492] LustreError: 328719:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10339.556267] LustreError: 328719:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10339.603280] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10343.456301] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10344.931164] LustreError: 328720:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10350.066655] LustreError: 328719:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10350.897486] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10351.231018] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10354.744564] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10361.391352] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10367.191621] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10374.297698] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10374.759766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10380.642349] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10387.642038] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10393.073130] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10393.220856] Lustre: lustre-OST0001: new disk, initializing [10393.226661] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10393.233694] Lustre: 331551:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10393.311834] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10398.311849] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10402.426740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10402.431995] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10402.470607] LustreError: 332120:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10403.489155] LustreError: 332125:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10403.512633] LustreError: 332125:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10404.515330] LustreError: 332120:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10406.565558] LustreError: 332120:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10406.580472] LustreError: 332120:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10408.020711] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10408.325763] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10410.720793] LustreError: 332125:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10410.748789] LustreError: 332125:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10413.477668] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9b7d50a66000.ost_server_uuid 50 [10414.760866] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9b7d50a66000.ost_server_uuid in FULL state after 0 sec [10418.921249] LustreError: 332120:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10418.932562] LustreError: 332120:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10419.891547] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9b7d50a66000.ost_server_uuid 50 [10421.359483] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9b7d50a66000.ost_server_uuid in FULL state after 0 sec [10423.111731] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10438.625986] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10438.631379] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10438.641370] Lustre: Skipped 1 previous similar message [10438.647056] Lustre: Skipped 1 previous similar message [10442.745885] Lustre: server umount lustre-OST0001 complete [10443.762606] LustreError: 330536:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10443.786529] LustreError: 330536:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10448.867261] LustreError: 330536:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10448.898452] LustreError: 330536:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10451.939560] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10451.946399] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10451.961546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10451.971546] Lustre: Skipped 2 previous similar messages [10455.562826] Lustre: server umount lustre-OST0000 complete [10465.159360] Lustre: server umount lustre-MDT0000 complete [10468.323759] LustreError: 328700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786822643 with bad export cookie 10591621970188828202 [10468.328184] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10468.341424] LustreError: 328700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10468.609065] Lustre: server umount lustre-MDT0001 complete [10479.828830] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [10482.910912] Key type lgssc unregistered [10483.188610] LNet: 334242:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10483.200905] LNetError: 334242:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10483.217690] LNet: Removed LNI 192.168.206.114@tcp [10484.234231] Key type .llcrypt unregistered [10484.236973] Key type ._llcrypt unregistered [10507.337784] Key type ._llcrypt registered [10507.340426] Key type .llcrypt registered [10507.473331] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [10508.415679] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10508.428141] alg: No test for adler32 (adler32-zlib) [10509.488482] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [10509.747651] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [10511.495201] Key type lgssc registered [10512.647419] Lustre: Echo OBD driver; http://www.lustre.org/ [10519.683925] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10519.713987] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10521.224739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10526.008922] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10530.785432] LustreError: 335676:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10530.812734] LustreError: 335676:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10531.808058] LustreError: 335677:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10532.708800] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10536.928946] LustreError: 335676:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10539.055266] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10539.521789] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10543.624280] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10552.498306] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10552.772257] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10558.377604] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10565.111454] LustreError: 337357:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10565.312505] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10566.194155] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10572.032640] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10572.213150] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10577.422758] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10578.388241] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10585.706976] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10592.849779] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10593.105463] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10700.259273] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10700.267597] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10700.283695] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10700.285186] LustreError: Skipped 1 previous similar message [10700.296638] Lustre: Skipped 1 previous similar message [10705.392540] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10705.400324] Lustre: Skipped 1 previous similar message [10706.455614] Lustre: server umount lustre-OST0001 complete [10710.497063] LustreError: 337357:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10710.515405] LustreError: 337357:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [10713.571285] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10713.581727] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10713.593591] Lustre: Skipped 1 previous similar message [10713.598668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10714.599143] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10714.609336] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10715.623820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10715.629518] Lustre: Skipped 2 previous similar messages [10720.743811] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10720.744219] LustreError: 338651:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10720.752236] Lustre: Skipped 1 previous similar message [10720.775190] LustreError: 338651:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10726.879968] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10727.006880] Lustre: server umount lustre-OST0000 complete [10730.989561] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10730.998118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10731.004977] Lustre: Skipped 1 previous similar message [10731.023788] Lustre: Skipped 3 previous similar messages [10736.695910] Lustre: server umount lustre-MDT0000 complete [10740.386615] LustreError: 335657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786822915 with bad export cookie 5244464027330523014 [10740.388796] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10740.399891] LustreError: 335657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10740.833575] Lustre: server umount lustre-MDT0001 complete [10751.735274] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [10754.226103] Key type lgssc unregistered [10754.524892] LNet: 340625:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10754.535571] LNetError: 340625:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10754.556221] LNet: Removed LNI 192.168.206.114@tcp [10755.421131] Key type .llcrypt unregistered [10755.427809] Key type ._llcrypt unregistered [10801.340721] Key type ._llcrypt registered [10801.344287] Key type .llcrypt registered [10801.424335] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [10802.526374] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10802.559783] alg: No test for adler32 (adler32-zlib) [10803.606120] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [10803.881930] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [10805.567153] Key type lgssc registered [10806.511197] Lustre: Echo OBD driver; http://www.lustre.org/ [10814.642518] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10814.662592] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10816.007799] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10820.114649] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10825.187277] LustreError: 342917:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10825.199482] LustreError: 342917:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10827.581172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10827.974698] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10832.335170] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10834.954080] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10841.270524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10841.630545] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10847.333717] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10852.903312] LustreError: 344408:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10855.415934] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10855.621192] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10860.786465] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10860.792811] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10861.986056] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10870.990932] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10879.227090] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10894.305616] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10894.311450] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10894.335735] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10896.868119] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10896.871107] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10896.881786] Lustre: Skipped 2 previous similar messages [10896.885309] Lustre: Skipped 2 previous similar messages [10898.263143] Lustre: server umount lustre-MDT0000 complete [10901.814740] LustreError: 342898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786823076 with bad export cookie 14193028696639858587 [10901.819179] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10901.833976] LustreError: 342898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10901.995463] LustreError: 343972:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10902.004758] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10902.008063] LustreError: 343972:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10902.020968] Lustre: Skipped 1 previous similar message [10902.165671] Lustre: server umount lustre-MDT0001 complete [10911.857475] Lustre: server umount lustre-OST0000 complete [10915.625033] Lustre: server umount lustre-OST0001 complete [10932.531672] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [10943.507470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10943.846660] LustreError: 348191:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10943.857028] LustreError: 348191:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10943.912989] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10948.011628] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10949.090655] LustreError: 348192:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10955.544635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10955.945606] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10959.705786] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10962.116636] Lustre: 349330:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10968.601213] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10974.503628] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10981.165155] LustreError: 349685:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10981.189096] LustreError: 349685:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10982.886202] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10983.303069] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10983.311307] Lustre: Skipped 1 previous similar message [10988.414523] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10988.417114] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10989.692179] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [10997.575846] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11001.200242] Lustre: 351203:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11006.945868] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11006.948343] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11006.977800] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11006.980302] Lustre: Skipped 1 previous similar message [11008.995282] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11011.816638] Lustre: server umount lustre-MDT0000 complete [11014.118785] LustreError: 349284:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11014.133272] LustreError: 349284:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11016.048971] LustreError: 348172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786823190 with bad export cookie 14193028696639860288 [11016.054713] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11016.058799] LustreError: 348172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11016.434475] Lustre: server umount lustre-MDT0001 complete [11030.549264] Lustre: server umount lustre-OST0000 complete [11043.543554] Lustre: server umount lustre-OST0001 complete [11046.808157] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11048.749308] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11050.506326] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 15:47:03 (1786823223) [11072.673895] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [11085.516414] loop0: detected capacity change from 0 to 1099511627776 [11086.804978] SGI XFS with ACLs, security attributes, quota, no debug enabled [11086.814266] *** VALIDATE xfs *** [11086.834494] XFS (loop0): Mounting V5 Filesystem [11086.886227] XFS (loop0): Ending clean mount [11088.877404] loop1: detected capacity change from 0 to 18691697672192 [11195.352696] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [11204.461489] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [11238.869105] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11239.148876] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11239.170321] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11239.241198] Lustre: lustre-MDT0000: new disk, initializing [11239.288154] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11239.305040] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11243.304393] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11252.895893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11253.065123] Lustre: 356338:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11253.216801] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11253.227212] Lustre: Skipped 1 previous similar message [11253.333295] Lustre: lustre-MDT0001: new disk, initializing [11253.583236] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11253.600940] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11258.839465] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11269.012966] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11274.377981] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11280.288309] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11280.677939] Lustre: lustre-OST0000: new disk, initializing [11280.686070] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11280.696392] Lustre: 358305:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11282.461621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11282.478850] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11282.622631] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11286.722962] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11296.089182] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11300.138797] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11300.334441] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11304.564477] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11304.749591] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11308.000766] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11308.007164] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11308.025643] Lustre: Skipped 3 previous similar messages [11308.028822] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11308.031956] Lustre: Skipped 3 previous similar messages [11312.381534] Lustre: server umount lustre-OST0000 complete [11318.752788] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11318.772105] Lustre: Skipped 1 previous similar message [11318.778384] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11318.780907] Lustre: Skipped 2 previous similar messages [11322.762149] Lustre: server umount lustre-MDT0000 complete [11323.887675] LustreError: 356349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11323.938311] LustreError: 356349:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11326.283319] LustreError: 356330:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786823501 with bad export cookie 14193028696639861842 [11326.300902] LustreError: 356330:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11326.301245] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11326.674498] Lustre: server umount lustre-MDT0001 complete [11335.182385] XFS (loop0): Unmounting Filesystem [11339.501953] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 15:51:52 (1786823512) [11346.500742] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11347.248913] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11347.256125] Lustre: Skipped 2 previous similar messages [11350.704788] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11359.958216] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11365.193792] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11372.962565] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11378.918093] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11386.258270] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11393.558523] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11400.782039] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11415.523938] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11415.527513] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11415.527688] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11415.527694] LustreError: Skipped 1 previous similar message [11415.539299] Lustre: Skipped 2 previous similar messages [11415.561025] Lustre: Skipped 1 previous similar message [11420.376342] Lustre: server umount lustre-OST0000 complete [11426.786239] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11432.932076] Lustre: server umount lustre-MDT0000 complete [11436.002635] LustreError: 361592:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11436.016203] LustreError: 361592:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [11437.632099] LustreError: 360844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786823612 with bad export cookie 14193028696639862724 [11437.643229] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11437.668643] LustreError: 360844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11438.151447] Lustre: server umount lustre-MDT0001 complete [11450.548798] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [11453.513906] Key type lgssc unregistered [11453.779488] LNet: 364499:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11453.786246] LNetError: 364499:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11453.803153] LNet: Removed LNI 192.168.206.114@tcp [11454.524165] Key type .llcrypt unregistered [11454.526684] Key type ._llcrypt unregistered [11480.282155] Key type ._llcrypt registered [11480.285559] Key type .llcrypt registered [11480.364936] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [11481.457668] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11481.482988] alg: No test for adler32 (adler32-zlib) [11482.559115] Lustre: Lustre: Build Version: 2.17.57_3_g67c4c6e [11482.784086] LNet: Added LNI 192.168.206.114@tcp [8/256/0/180] [11484.423470] Key type lgssc registered [11485.499850] Lustre: Echo OBD driver; http://www.lustre.org/ [11493.339785] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11493.375748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11494.927526] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11498.310664] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11505.129608] LustreError: 366067:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11505.154239] LustreError: 366067:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11508.359359] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11508.817847] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11512.739968] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11520.728770] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11526.415791] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11534.048331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11534.521397] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11541.111947] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11548.037800] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11559.394782] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11559.409179] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11559.416300] Lustre: Skipped 1 previous similar message [11560.863573] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.14@tcp (stopping) [11564.518063] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11564.526943] Lustre: Skipped 1 previous similar message [11564.963382] Lustre: server umount lustre-MDT0001 complete [11565.960210] LustreError: 366061:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.14@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11569.637676] LustreError: 367308:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11569.667970] LustreError: 367308:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11574.757943] LustreError: 366062:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11574.793122] LustreError: 366062:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11575.210868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11575.369714] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11575.381985] Lustre: Found index 1 for lustre-MDT0001, updating log [11575.422268] Lustre: 366053:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11575.759300] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11580.193737] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11580.919246] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11582.449683] LustreError: 369640:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11582.462599] LustreError: 369640:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11582.473647] LustreError: 366058:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11582.528658] LustreError: 369645:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.114@tcp: cfg command failed: rc = -17 [11582.548017] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11582.548017] [11591.138179] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11591.139738] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11591.158474] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11591.177250] Lustre: Skipped 2 previous similar messages [11591.185985] Lustre: Skipped 1 previous similar message [11595.217222] Lustre: server umount lustre-MDT0001 complete [11596.267458] LustreError: 366067:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11596.291345] LustreError: 366067:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11605.200239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11605.272279] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11605.275719] Lustre: Found index 1 for lustre-MDT0001, updating log [11605.297472] Lustre: 366055:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11605.570464] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11609.240572] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11610.612764] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11612.664516] LustreError: 370742:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11612.673617] LustreError: 370742:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11612.681544] LustreError: 366058:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11612.685895] LustreError: 366058:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11612.735278] LustreError: 370745:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11612.742700] LustreError: 370745:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11612.747517] LustreError: 370745:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.114@tcp: cfg command failed: rc = -17 [11612.753082] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.114@tcp [11612.753082] [11917.793181] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11917.814214] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11917.828462] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11917.871561] Lustre: Skipped 2 previous similar messages [11921.017821] Lustre: server umount lustre-MDT0001 complete [11922.918798] LustreError: 366063:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11922.945914] LustreError: 366063:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [11932.615602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11932.793617] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11932.800077] Lustre: Found index 1 for lustre-MDT0001, updating log [11932.867114] Lustre: 366055:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11933.271525] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11938.301890] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11938.343170] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [11942.381293] LustreError: 379529:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11942.389384] LustreError: 379529:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11942.409962] LustreError: 366058:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11942.414389] LustreError: 366058:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11942.466210] LustreError: 379532:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11942.483085] LustreError: 379532:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.114@tcp: cfg command failed: rc = -114 [11942.494563] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11942.494563] [12260.833174] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12260.845337] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12260.863163] Lustre: Skipped 1 previous similar message [12260.867602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12260.878373] Lustre: Skipped 1 previous similar message [12274.143440] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12274.390833] Lustre: server umount lustre-MDT0000 complete [12278.445299] LustreError: 366046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786824453 with bad export cookie 1741018531353691564 [12278.456305] LustreError: 366046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12278.459223] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12279.266929] LustreError: 366062:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12279.269201] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12279.298575] LustreError: 366062:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [12279.326191] Lustre: Skipped 2 previous similar messages [12279.337501] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12279.343965] Lustre: Skipped 9 previous similar messages [12285.225982] Lustre: server umount lustre-MDT0001 complete [12291.679073] Lustre: server umount lustre-OST0000 complete [12302.520335] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [12314.788335] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [12349.901725] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 16:08:42 (1786824522) [12355.938951] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12356.119648] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12356.172535] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12356.258206] Lustre: lustre-MDT0000: new disk, initializing [12356.328297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12356.359062] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12360.304904] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [12370.316552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12370.406224] Lustre: 390162:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12370.447920] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12370.450611] Lustre: Skipped 1 previous similar message [12370.537272] Lustre: lustre-MDT0001: new disk, initializing [12370.617931] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12370.657505] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12370.668946] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12375.674975] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [12385.156393] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12390.810801] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12397.191551] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12397.410324] Lustre: lustre-OST0000: new disk, initializing [12397.414787] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12397.426854] Lustre: 392129:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12397.505739] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12403.393607] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_default_debug -1 all [12404.277315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12404.317943] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12404.379773] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12414.033059] Lustre: DEBUG MARKER: oleg614-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12424.162467] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12424.172468] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12424.181322] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12424.184825] Lustre: Skipped 1 previous similar message [12424.601870] Lustre: server umount lustre-MDT0000 complete [12425.187824] LustreError: 390174:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12425.217359] LustreError: 390174:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12428.502128] LustreError: 390154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786824603 with bad export cookie 1741018531353693510 [12428.509078] LustreError: 390968:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12428.511335] LustreError: MGC192.168.206.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12428.524964] LustreError: 390154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12428.868275] Lustre: server umount lustre-MDT0001 complete [12443.081642] Lustre: server umount lustre-OST0000 complete [12475.476599] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing set_hostid [12487.241258] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing load_modules_local [12518.618735] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12236 sec ========= 16:11:31 (1786824691) [12520.534534] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 16:11:32 (1786824692) === [12523.753618] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 16:11:36 (1786824696) === [12562.574124] Lustre: DEBUG MARKER: oleg614-server.virtnet: executing unload_modules_local [12565.176702] Key type lgssc unregistered [12565.516419] LNet: 398498:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12565.524863] LNetError: 398498:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12565.551086] LNet: Removed LNI 192.168.206.114@tcp [12566.403494] Key type .llcrypt unregistered [12566.407213] Key type ._llcrypt unregistered