[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 487987291 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002355] x2apic enabled [ 0.003007] Switched APIC routing to physical x2apic. [ 0.004010] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007023] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008007] pid_max: default: 32768 minimum: 301 [ 0.009102] LSM: Security Framework initializing [ 0.010048] Yama: becoming mindful. [ 0.011034] SELinux: Initializing. [ 0.012056] *** VALIDATE selinux *** [ 0.020353] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024553] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025128] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026097] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027104] *** VALIDATE tmpfs *** [ 0.028422] *** VALIDATE proc *** [ 0.029268] *** VALIDATE cgroup *** [ 0.030009] *** VALIDATE cgroup2 *** [ 0.031273] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.032157] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.033007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.034026] Spectre V2 : User space: Vulnerable [ 0.035009] Speculative Store Bypass: Vulnerable [ 0.038343] debug: unmapping init [mem 0xffffffffbb859000-0xffffffffbb860fff] [ 0.040188] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.041671] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.042023] ... version: 2 [ 0.043012] ... bit width: 48 [ 0.044012] ... generic registers: 4 [ 0.045010] ... value mask: 0000ffffffffffff [ 0.046017] ... max period: 00007fffffffffff [ 0.047015] ... fixed-purpose events: 3 [ 0.048014] ... event mask: 000000070000000f [ 0.049303] rcu: Hierarchical SRCU implementation. [ 0.051459] smp: Bringing up secondary CPUs ... [ 0.052584] x86: Booting SMP configuration: [ 0.053025] .... node #0, CPUs: #1 #2 #3 [ 0.056262] smp: Brought up 1 node, 4 CPUs [ 0.058013] smpboot: Max logical packages: 1 [ 0.059025] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.131459] node 0 deferred pages initialised in 70ms [ 0.134008] devtmpfs: initialized [ 0.134846] x86/mm: Memory block size: 128MB [ 0.136000] gcov: version magic: 0x41383552 [ 0.137257] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.140097] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.141280] pinctrl core: initialized pinctrl subsystem [ 0.143164] [ 0.143545] ************************************************************* [ 0.145011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.147008] ** ** [ 0.148009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.150008] ** ** [ 0.151010] ** This means that this kernel is built to expose internal ** [ 0.152007] ** IOMMU data structures, which may compromise security on ** [ 0.154007] ** your system. ** [ 0.155007] ** ** [ 0.156010] ** If you see this message and you are not debugging the ** [ 0.158007] ** kernel, report this immediately to your vendor! ** [ 0.159007] ** ** [ 0.161009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.162006] ************************************************************* [ 0.163582] NET: Registered protocol family 16 [ 0.165283] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.167035] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.168054] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.170355] cpuidle: using governor menu [ 0.171888] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.173361] PCI: Using configuration type 1 for base access [ 0.175126] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.182079] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.184038] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.186217] cryptd: max_cpu_qlen set to 1000 [ 0.189183] ACPI: Added _OSI(Module Device) [ 0.190014] ACPI: Added _OSI(Processor Device) [ 0.192012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.193014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.198323] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.204514] ACPI: Interpreter enabled [ 0.206058] ACPI: PM: (supports S0 S3 S4 S5) [ 0.208015] ACPI: Using IOAPIC for interrupt routing [ 0.209093] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.212405] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.222948] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.225040] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.228019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.232111] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.236756] acpiphp: Slot [2] registered [ 0.238115] acpiphp: Slot [5] registered [ 0.240091] acpiphp: Slot [6] registered [ 0.241101] acpiphp: Slot [7] registered [ 0.243134] acpiphp: Slot [8] registered [ 0.244079] acpiphp: Slot [9] registered [ 0.245130] acpiphp: Slot [10] registered [ 0.247141] acpiphp: Slot [3] registered [ 0.248107] acpiphp: Slot [4] registered [ 0.250151] acpiphp: Slot [11] registered [ 0.251106] acpiphp: Slot [12] registered [ 0.253104] acpiphp: Slot [13] registered [ 0.254132] acpiphp: Slot [14] registered [ 0.256107] acpiphp: Slot [15] registered [ 0.257094] acpiphp: Slot [16] registered [ 0.259096] acpiphp: Slot [17] registered [ 0.260076] acpiphp: Slot [18] registered [ 0.261135] acpiphp: Slot [19] registered [ 0.262109] acpiphp: Slot [20] registered [ 0.264206] acpiphp: Slot [21] registered [ 0.266101] acpiphp: Slot [22] registered [ 0.267081] acpiphp: Slot [23] registered [ 0.269168] acpiphp: Slot [24] registered [ 0.270166] acpiphp: Slot [25] registered [ 0.272094] acpiphp: Slot [26] registered [ 0.274103] acpiphp: Slot [27] registered [ 0.275098] acpiphp: Slot [28] registered [ 0.277092] acpiphp: Slot [29] registered [ 0.278091] acpiphp: Slot [30] registered [ 0.279179] acpiphp: Slot [31] registered [ 0.281063] PCI host bridge to bus 0000:00 [ 0.282013] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.285023] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.287028] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.289022] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.291020] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.294026] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.296208] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.299055] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.301888] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.308713] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.313084] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.315021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.317024] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.320020] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.322483] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.327000] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.329045] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.332845] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.337000] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.351014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.356862] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.361490] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.381018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.388023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.405029] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.429022] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.434014] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.438014] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.449013] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.456912] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.461012] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.464013] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.476014] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.484852] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.493016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.501017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.510014] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.520501] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.529020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.536017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.556014] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.565808] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.570013] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.574023] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.584015] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.592092] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.594359] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.597347] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.599270] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.601151] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.604036] iommu: Default domain type: Passthrough [ 0.605418] SCSI subsystem initialized [ 0.607171] ACPI: bus type USB registered [ 0.609112] usbcore: registered new interface driver usbfs [ 0.611083] usbcore: registered new interface driver hub [ 0.612070] usbcore: registered new device driver usb [ 0.614193] pps_core: LinuxPPS API ver. 1 registered [ 0.616025] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.619071] PTP clock support registered [ 0.621065] EDAC MC: Ver: 3.0.0 [ 0.622408] PCI: Using ACPI for IRQ routing [ 0.623688] NetLabel: Initializing [ 0.625011] NetLabel: domain hash size = 128 [ 0.626005] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.627057] NetLabel: unlabeled traffic allowed by default [ 0.628111] vgaarb: loaded [ 0.630085] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.631009] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.637333] clocksource: Switched to clocksource kvm-clock [ 0.749560] VFS: Disk quotas dquot_6.6.0 [ 0.751218] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.753612] *** VALIDATE ramfs *** [ 0.754793] *** VALIDATE hugetlbfs *** [ 0.756329] pnp: PnP ACPI init [ 0.758556] pnp: PnP ACPI: found 6 devices [ 0.790335] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.793671] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.795688] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.797473] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.799530] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.801698] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.804391] NET: Registered protocol family 2 [ 0.806725] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.811277] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.814670] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.818871] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.821420] TCP: Hash tables configured (established 65536 bind 65536) [ 0.823368] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.825285] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.827253] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.829186] NET: Registered protocol family 1 [ 0.831235] RPC: Registered named UNIX socket transport module. [ 0.833451] RPC: Registered udp transport module. [ 0.835145] RPC: Registered tcp transport module. [ 0.836949] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.839171] NET: Registered protocol family 44 [ 0.841109] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.843102] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.844639] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.846219] PCI: CLS 0 bytes, default 64 [ 0.847502] Unpacking initramfs... [ 2.197073] debug: unmapping init [mem 0xffff8e6ffcc54000-0xffff8e6ffffbffff] [ 2.199888] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.201819] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.204216] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.682291] Initialise system trusted keyrings [ 2.683865] Key type blacklist registered [ 2.685575] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.694266] zbud: loaded [ 2.697200] *** VALIDATE nfs *** [ 2.698517] *** VALIDATE nfs4 *** [ 2.700251] pstore: using deflate compression [ 2.703660] Platform Keyring initialized [ 2.803540] NET: Registered protocol family 38 [ 2.805244] Key type asymmetric registered [ 2.806774] Asymmetric key parser 'x509' registered [ 2.808220] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.810698] io scheduler mq-deadline registered [ 2.812193] io scheduler kyber registered [ 2.813852] io scheduler bfq registered [ 2.816078] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.818939] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.821595] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.824286] ACPI: Power Button [PWRF] [ 2.829342] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.835904] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.855850] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.862923] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.882860] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.913364] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.942810] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.947605] Non-volatile memory driver v1.3 [ 2.949348] Linux agpgart interface v0.103 [ 2.979741] virtio_blk virtio1: [vda] 150616 512-byte logical blocks (77.1 MB/73.5 MiB) [ 2.982902] vda: detected capacity change from 0 to 77115392 [ 2.998516] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.001840] vdb: detected capacity change from 0 to 1073741824 [ 3.016154] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.017934] vdc: detected capacity change from 0 to 2621440000 [ 3.029943] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.032443] vdd: detected capacity change from 0 to 2621440000 [ 3.046428] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.049393] vde: detected capacity change from 0 to 4294967296 [ 3.067535] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.070547] vdf: detected capacity change from 0 to 4294967296 [ 3.076821] libphy: Fixed MDIO Bus: probed [ 3.086033] usbcore: registered new interface driver usbserial_generic [ 3.087867] usbserial: USB Serial support registered for generic [ 3.089638] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.092615] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.093786] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.095438] mousedev: PS/2 mouse device common for all mice [ 3.097366] rtc_cmos 00:05: RTC can wake from S4 [ 3.099143] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.100651] rtc_cmos 00:05: registered as rtc0 [ 3.106102] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.106503] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.109338] intel_pstate: CPU model not supported [ 3.112217] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.116219] hid: raw HID events driver (C) Jiri Kosina [ 3.118389] usbcore: registered new interface driver usbhid [ 3.120344] usbhid: USB HID core driver [ 3.121944] drop_monitor: Initializing network drop monitor service [ 3.124301] Initializing XFRM netlink socket [ 3.126518] NET: Registered protocol family 10 [ 3.129410] Segment Routing with IPv6 [ 3.130701] NET: Registered protocol family 17 [ 3.132864] mpls_gso: MPLS GSO support [ 3.136732] RAS: Correctable Errors collector initialized. [ 3.138552] AVX version of gcm_enc/dec engaged. [ 3.139400] AES CTR mode by8 optimization enabled [ 3.201318] sched_clock: Marking stable (3201298806, 0)->(4024771957, -823473151) [ 3.203766] registered taskstats version 1 [ 3.205551] Loading compiled-in X.509 certificates [ 3.206969] zswap: loaded using pool lzo/zbud [ 3.227535] Key type big_key registered [ 3.237352] Key type encrypted registered [ 3.238533] ima: No TPM chip found, activating TPM-bypass! [ 3.239816] ima: Allocated hash algorithm: sha1 [ 3.240705] ima: No architecture policies found [ 3.241882] evm: Initialising EVM extended attributes: [ 3.243295] evm: security.selinux [ 3.243977] evm: security.ima [ 3.244536] evm: security.capability [ 3.245312] evm: HMAC attrs: 0x1 [ 3.246831] rtc_cmos 00:05: setting system clock to 2026-09-30 14:36:50 UTC (1790779010) [ 3.252604] debug: unmapping init [mem 0xffffffffbc803000-0xffffffffbc9fffff] [ 3.255825] debug: unmapping init [mem 0xffffffffbb582000-0xffffffffbb858fff] [ 3.263101] Write protecting the kernel read-only data: 28672k [ 3.265758] debug: unmapping init [mem 0xffffffffb9c03000-0xffffffffb9dfffff] [ 3.268057] debug: unmapping init [mem 0xffffffffba514000-0xffffffffba5fffff] [ 3.295627] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.301518] systemd[1]: Detected virtualization kvm. [ 3.303251] systemd[1]: Detected architecture x86-64. [ 3.304900] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.328125] systemd[1]: No hostname configured. [ 3.329949] systemd[1]: Set hostname to . [ 3.331583] random: systemd: uninitialized urandom read (16 bytes read) [ 3.333225] systemd[1]: Initializing machine ID from random generator. [ 3.447691] random: systemd: uninitialized urandom read (16 bytes read) [ 3.449754] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.452954] random: systemd: uninitialized urandom read (16 bytes read) [ 3.454703] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.458087] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Swap. [ OK ] Reached target Paths. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Timers. Starting Setup Virtual Console... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... Starting Journal Service... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.957485] device-mapper: uevent: version 1.0.3 [ 3.959750] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.674210] virtio_net virtio0 ens2: renamed from eth0 [ 4.683135] random: fast init done [ 4.692715] scsi host0: ata_piix [ 4.728030] scsi host1: ata_piix [ 4.729580] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.731822] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.755287] random: crng init done [ 9.760165] random: 7 urandom warning(s) missed due to ratelimiting [ 9.910954] dracut-initqueue[590]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 11.454552] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 14.258884] printk: systemd: 26 output lines suppressed due to ratelimiting [ 15.146830] SELinux: Disabled at runtime. [ 15.298193] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 15.324334] systemd[1]: Detected virtualization kvm. [ 15.326907] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.185869] systemd[1]: initrd-switch-root.service: Succeeded. [ 17.191771] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 17.200782] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 17.208916] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 17.213716] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 17.223695] systemd[1]: Starting Journal Service... Starting Journal Service... [ 17.235558] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Mounting POSIX Message Queue File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. Starting Apply Kernel Variables... [ 17.338856] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on udev Control Socket. Mounting Kernel Debug File System... [ OK ] Created slice User and Session Slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target Slices. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice system-getty.slice. Starting Remount Root and Kernel File Systems... [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target rpc_pipefs.target. Mounting Huge Pages File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Reached target Swap. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 18.647319] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 19.742905] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 19.775797] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 20.389314] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 20.489463] EDAC sbridge: Ver: 1.1.2 [ 24.287846] Key type dns_resolver registered [* ] A start job is running for Configur…-only root support (7s / no limit)[ 25.149084] NFS: Registering the id_resolver key type [ 25.151167] Key type id_resolver registered [ 25.152973] Key type id_legacy registered [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (8s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Login Service. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg243-server login: [ 53.290734] hrtimer: interrupt took 5505951 ns [ 83.208627] libcfs: loading out-of-tree module taints kernel. [ 83.241311] Key type ._llcrypt registered [ 83.242745] Key type .llcrypt registered [ 83.367436] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 99.308660] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 100.795862] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 100.826395] alg: No test for adler32 (adler32-zlib) [ 102.561763] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 104.153809] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 106.192643] Key type lgssc registered [ 108.545462] Lustre: Echo OBD driver; http://www.lustre.org/ [ 128.171126] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 158.633338] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 171.192985] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 171.235736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 172.571546] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 172.620036] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 172.735420] Lustre: lustre-MDT0000: new disk, initializing [ 172.829520] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 172.845632] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 177.165674] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 181.738879] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 190.270621] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 190.487861] Lustre: lustre-OST0000: new disk, initializing [ 190.492437] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 190.499213] Lustre: Skipped 1 previous similar message [ 190.502870] Lustre: 6954:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 190.570814] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 192.855696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 192.867201] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 192.899328] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 195.673817] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 209.157124] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 209.327552] Lustre: lustre-OST0001: new disk, initializing [ 209.331224] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 209.334533] Lustre: 8017:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 209.404288] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 215.689930] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 219.209682] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 219.218138] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 219.241865] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 226.465590] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 231.697807] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 237.320843] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing check_logdir /tmp/testlogs/ [ 242.922966] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing yml_node [ 246.730902] Lustre: DEBUG MARKER: Client: 2.17.59.2 [ 249.281935] Lustre: DEBUG MARKER: MDS: 2.17.59.2 [ 251.499571] Lustre: DEBUG MARKER: OSS: 2.17.59.2 [ 253.000283] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Sep 30 10:40:58 EDT 2026 [ 269.723239] Lustre: DEBUG MARKER: only running tests: 0-31 [ 271.238334] Lustre: DEBUG MARKER: excepting tests: 21e 32b 32c 32newtarball [ 272.743345] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 290.785291] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 290.799171] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 290.810327] Lustre: Skipped 1 previous similar message [ 294.155373] Lustre: server umount lustre-MDT0000 complete [ 297.329303] LustreError: 5936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790779304 with bad export cookie 6242077729268594071 [ 297.356269] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 297.432582] Lustre: server umount lustre-OST0000 complete [ 301.025353] Lustre: server umount lustre-OST0001 complete [ 306.698936] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 314.142357] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 348.042345] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 348.292520] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 348.337094] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 348.420702] Lustre: lustre-MDT0000: new disk, initializing [ 348.479502] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 348.500813] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 353.110395] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 362.329565] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 369.018388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 369.269804] Lustre: lustre-OST0000: new disk, initializing [ 369.272343] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 369.274353] Lustre: Skipped 1 previous similar message [ 369.281096] Lustre: 14137:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 369.368465] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 370.825304] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 370.831972] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 370.853916] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 374.509879] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 382.941574] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 387.753578] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 387.940365] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 391.145391] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 391.155850] Lustre: Skipped 1 previous similar message [ 391.166217] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 395.969502] Lustre: server umount lustre-OST0000 complete [ 400.862293] Lustre: server umount lustre-MDT0000 complete [ 405.388179] Lustre: DEBUG MARKER: == conf-sanity test 0: single mount setup ================ 10:43:30 (1790779410) [ 412.702103] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 413.084961] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 418.284718] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 425.542824] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 432.396056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 432.755812] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 439.423529] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 446.853370] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 453.601781] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 453.613479] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 453.628953] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 458.722908] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 459.166583] Lustre: server umount lustre-OST0000 complete [ 463.196263] Lustre: server umount lustre-MDT0000 complete [ 474.135279] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 477.205833] Key type lgssc unregistered [ 477.640751] LNet: 18348:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 477.655391] LNetError: 18348:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 477.675835] LNet: Removed LNI 192.168.202.143@tcp [ 479.015705] Key type .llcrypt unregistered [ 479.017964] Key type ._llcrypt unregistered [ 503.458533] Key type ._llcrypt registered [ 503.460764] Key type .llcrypt registered [ 503.544565] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 504.913525] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 504.983807] alg: No test for adler32 (adler32-zlib) [ 506.135195] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 506.338353] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 508.048742] Key type lgssc registered [ 509.137305] Lustre: Echo OBD driver; http://www.lustre.org/ [ 515.742863] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 515.762386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 517.166681] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 520.837177] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 527.989463] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 534.274353] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 534.609806] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 540.244069] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 541.820400] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:35 to 0x240000400:65) [ 547.191034] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 562.672992] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 562.681109] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 562.688281] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 563.806446] Lustre: server umount lustre-OST0000 complete [ 567.570478] Lustre: server umount lustre-MDT0000 complete [ 575.953538] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 578.452140] Key type lgssc unregistered [ 578.660692] LNet: 22639:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 578.676951] LNetError: 22639:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 578.702830] LNet: Removed LNI 192.168.202.143@tcp [ 579.363489] Key type .llcrypt unregistered [ 579.370142] Key type ._llcrypt unregistered [ 599.933868] Key type ._llcrypt registered [ 599.935898] Key type .llcrypt registered [ 600.028450] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 600.991213] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 601.033621] alg: No test for adler32 (adler32-zlib) [ 602.148161] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 602.358466] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 604.041296] Key type lgssc registered [ 605.197259] Lustre: Echo OBD driver; http://www.lustre.org/ [ 611.595223] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 611.621146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 612.976760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 616.337637] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 622.970196] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 640.698521] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 641.271812] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 649.568880] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 657.390599] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 658.612622] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:67 to 0x240000400:97) [ 669.155689] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 669.171518] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 670.563802] Lustre: server umount lustre-OST0000 complete [ 674.938182] Lustre: server umount lustre-MDT0000 complete [ 685.697743] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 688.287794] Key type lgssc unregistered [ 688.609154] LNet: 26814:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 688.616900] LNetError: 26814:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 688.637563] LNet: Removed LNI 192.168.202.143@tcp [ 689.476918] Key type .llcrypt unregistered [ 689.478631] Key type ._llcrypt unregistered [ 708.282129] Key type ._llcrypt registered [ 708.283790] Key type .llcrypt registered [ 708.347799] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 708.987394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 709.002327] alg: No test for adler32 (adler32-zlib) [ 710.065690] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 710.304416] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 711.992765] Key type lgssc registered [ 712.981896] Lustre: Echo OBD driver; http://www.lustre.org/ [ 719.910737] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 719.955695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 721.374356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 725.804877] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 732.486670] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 739.322683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 739.757980] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 745.915434] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 746.270351] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 753.759671] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 766.435582] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 766.435755] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 766.467982] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 771.567043] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 776.673643] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 777.184145] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 777.372712] Lustre: server umount lustre-OST0000 complete [ 781.872736] Lustre: server umount lustre-MDT0000 complete [ 795.893993] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 798.941771] Key type lgssc unregistered [ 799.181802] LNet: 30639:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 799.191532] LNetError: 30639:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 799.211720] LNet: Removed LNI 192.168.202.143@tcp [ 800.212405] Key type .llcrypt unregistered [ 800.217280] Key type ._llcrypt unregistered [ 823.497524] Key type ._llcrypt registered [ 823.499592] Key type .llcrypt registered [ 823.603295] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 824.532479] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 824.606910] alg: No test for adler32 (adler32-zlib) [ 825.618388] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 825.810962] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 827.490525] Key type lgssc registered [ 828.849736] Lustre: Echo OBD driver; http://www.lustre.org/ [ 835.131628] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 835.152228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 836.617877] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 841.179186] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 848.675476] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 855.570503] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 855.947150] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 861.109717] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:131 to 0x240000400:161) [ 862.908913] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 869.908675] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 876.523079] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 876.533149] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 876.577877] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 881.634925] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 881.641796] Lustre: Skipped 1 previous similar message [ 881.839655] Lustre: server umount lustre-OST0000 complete [ 908.291791] Lustre: server umount lustre-MDT0000 complete [ 919.558167] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 922.274012] Key type lgssc unregistered [ 922.687952] LNet: 34668:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 922.692661] LNetError: 34668:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 923.757659] LNet: Removed LNI 192.168.202.143@tcp [ 924.551195] Key type .llcrypt unregistered [ 924.553538] Key type ._llcrypt unregistered [ 946.719965] Key type ._llcrypt registered [ 946.726190] Key type .llcrypt registered [ 946.803159] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 947.686601] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 947.716772] alg: No test for adler32 (adler32-zlib) [ 948.856735] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 949.048093] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 950.696663] Key type lgssc registered [ 951.713358] Lustre: Echo OBD driver; http://www.lustre.org/ [ 958.747185] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 958.758537] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 960.253588] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 965.428384] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 972.117224] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 979.244507] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 979.508764] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 985.642284] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 985.651556] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:163 to 0x240000400:193) [ 993.939856] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1000.678254] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.43@tcp (stopping) [ 1000.936280] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1005.737656] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.43@tcp (stopping) [ 1005.754232] Lustre: Skipped 1 previous similar message [ 1006.690096] Lustre: server umount lustre-MDT0000 complete [ 1019.714783] LustreError: 36073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 192.168.202.43@tcp arrived at 1790780026 with bad export cookie 201632346628462878 [ 1019.725756] LustreError: 36073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1022.781545] LustreError: 36075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790780030 with bad export cookie 201632346628462297 [ 1022.801155] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1022.806805] LustreError: 36075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1037.280124] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1037.508481] Lustre: server umount lustre-OST0000 complete [ 1050.389466] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 1053.297595] Key type lgssc unregistered [ 1053.556994] LNet: 38689:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1053.561951] LNetError: 38689:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1053.576652] LNet: Removed LNI 192.168.202.143@tcp [ 1054.054478] Key type .llcrypt unregistered [ 1054.056122] Key type ._llcrypt unregistered [ 1082.518131] Key type ._llcrypt registered [ 1082.522125] Key type .llcrypt registered [ 1082.695625] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1083.970316] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1083.988853] alg: No test for adler32 (adler32-zlib) [ 1085.082434] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 1085.499480] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 1087.208281] Key type lgssc registered [ 1088.406577] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1096.085663] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1096.105510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1111.648406] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1116.769091] LustreError: 40103:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.143@tcp: failed processing log, type 4: rc = -110 [ 1143.520332] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1149.937144] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1158.326569] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1326.323476] Lustre: server umount lustre-OST0000 complete [ 1339.230737] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 1341.672331] Key type lgssc unregistered [ 1341.893374] LNet: 41624:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1341.900940] LNetError: 41624:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1341.926672] LNet: Removed LNI 192.168.202.143@tcp [ 1342.807594] Key type .llcrypt unregistered [ 1342.809906] Key type ._llcrypt unregistered [ 1369.018769] Key type ._llcrypt registered [ 1369.022536] Key type .llcrypt registered [ 1369.195715] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1370.249833] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1370.400609] alg: No test for adler32 (adler32-zlib) [ 1371.534918] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 1371.796445] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 1373.464694] Key type lgssc registered [ 1375.338165] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1384.528296] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1384.543586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1385.892077] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1391.359355] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1398.152454] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1405.657100] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1406.031361] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1411.185484] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:195 to 0x240000400:225) [ 1412.754686] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1420.431917] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1426.914558] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1426.938540] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1426.957261] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1428.891572] Lustre: server umount lustre-OST0000 complete [ 1433.648229] Lustre: server umount lustre-MDT0000 complete [ 1445.486550] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 1448.267313] Key type lgssc unregistered [ 1448.542750] LNet: 45468:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1448.549936] LNetError: 45468:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1449.577335] LNet: Removed LNI 192.168.202.143@tcp [ 1450.620726] Key type .llcrypt unregistered [ 1450.623325] Key type ._llcrypt unregistered [ 1475.969531] Key type ._llcrypt registered [ 1475.971943] Key type .llcrypt registered [ 1476.159238] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1477.309430] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1477.730551] alg: No test for adler32 (adler32-zlib) [ 1478.809188] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 1479.081515] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 1480.760231] Key type lgssc registered [ 1482.217540] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1490.830765] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1490.863245] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1506.464872] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1511.584320] LustreError: 46883:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.143@tcp: failed processing log, type 4: rc = -110 [ 1538.272932] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1545.262469] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1554.327684] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1561.401388] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1561.991533] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1562.116683] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:195 to 0x240000400:257) [ 1568.476052] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1576.971592] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1582.562320] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1582.571909] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1582.591171] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1583.586662] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1588.705740] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1593.312138] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1593.495393] Lustre: server umount lustre-OST0000 complete [ 1621.745321] Lustre: server umount lustre-MDT0000 complete [ 1632.597379] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 1636.363820] Key type lgssc unregistered [ 1636.789925] LNet: 49518:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1636.796150] LNetError: 49518:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1636.828022] LNet: Removed LNI 192.168.202.143@tcp [ 1638.087948] Key type .llcrypt unregistered [ 1638.097663] Key type ._llcrypt unregistered [ 1663.477273] Key type ._llcrypt registered [ 1663.479400] Key type .llcrypt registered [ 1663.640877] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1665.603802] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1665.649921] alg: No test for adler32 (adler32-zlib) [ 1666.813846] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 1667.223708] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 1668.944278] Key type lgssc registered [ 1670.819851] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1677.840941] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1677.882147] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1679.298641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1683.422818] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1691.867751] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1698.711491] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1699.307385] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1704.441165] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:195 to 0x240000400:289) [ 1706.696352] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1714.469417] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1735.138957] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1735.149994] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1735.161421] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1739.434520] Lustre: server umount lustre-OST0000 complete [ 1745.346432] Lustre: server umount lustre-MDT0000 complete [ 1756.650160] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 1759.787394] Key type lgssc unregistered [ 1760.056728] LNet: 53386:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1760.070290] LNetError: 53386:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1760.094866] LNet: Removed LNI 192.168.202.143@tcp [ 1760.890765] Key type .llcrypt unregistered [ 1760.892877] Key type ._llcrypt unregistered [ 1791.399789] Key type ._llcrypt registered [ 1791.402490] Key type .llcrypt registered [ 1791.591168] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1793.409546] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1793.424371] alg: No test for adler32 (adler32-zlib) [ 1794.766771] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 1795.030971] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 1796.664249] Key type lgssc registered [ 1797.741642] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1807.033383] Lustre: DEBUG MARKER: == conf-sanity test 5h: start mdt failure at mdt_fs_setup() ========================================================== 11:06:52 (1790780812) [ 1812.908748] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1812.975733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1814.861861] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1819.854514] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1826.698839] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1833.675932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1834.031091] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1840.227781] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:195 to 0x240000400:321) [ 1841.402355] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1850.389912] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1856.767987] Lustre: Failing over lustre-MDT0000 [ 1857.084475] Lustre: server umount lustre-MDT0000 complete [ 1867.219973] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1867.335722] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1867.644733] Lustre: *** cfs_fail_loc=135, val=0*** [ 1867.766465] LustreError: 57298:0:(obd_config.c:845:class_setup()) setup lustre-MDT0000 failed (-2) [ 1867.769759] LustreError: 57298:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.143@tcp: cfg command failed: rc = -2 [ 1867.774056] Lustre: cmd=cf003 0:lustre-MDT0000 1:lustre-MDT0000_UUID 2:0 3:lustre-MDT0000-mdtlov 4:f [ 1867.774056] [ 1867.778793] LustreError: MGC192.168.202.143@tcp: Confguration from log lustre-MDT0000 failed from MGS -2. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info [ 1867.787706] LustreError: 57270:0:(tgt_mount.c:1884:server_start_targets()) failed to start server lustre-MDT0000: -2 [ 1867.792156] LustreError: 57270:0:(tgt_mount.c:2613:server_fill_super()) Unable to start targets: -2 [ 1867.798459] LustreError: 57270:0:(tgt_mount.c:2128:server_put_super()) no obd lustre-MDT0000 [ 1867.801256] LustreError: 57270:0:(tgt_mount.c:116:server_deregister_mount()) lustre-MDT0000 not registered [ 1867.915146] Lustre: server umount lustre-MDT0000 complete [ 1867.922547] LustreError: 57270:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -2 [ 1875.844373] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1875.941105] Lustre: 54164:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790780867/real 1790780867] req@000000003757af79 x1877769770249472/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790780883 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1875.975063] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1875.976931] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1876.003779] LustreError: 55096:0:(mgc_request.c:614:do_requeue()) failed processing log: -5 [ 1876.477909] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1876.547886] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1881.185746] Lustre: 54165:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790780872/real 1790780872] req@000000005d9b91a9 x1877769770249728/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790780888 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1881.475495] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1884.710832] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects [ 1884.750388] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted. [ 1884.819110] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:195 to 0x240000400:353) [ 1886.702704] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1887.392171] Lustre: 54165:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790780878/real 1790780878] req@0000000055f5d55f x1877769770251264/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790780894 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1890.283672] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1890.291277] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1890.309126] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1891.948451] Lustre: server umount lustre-OST0000 complete [ 1896.341509] Lustre: server umount lustre-MDT0000 complete [ 1908.812860] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 1912.018950] Key type lgssc unregistered [ 1912.427425] LNet: 58881:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1912.438911] LNetError: 58881:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1912.457989] LNet: Removed LNI 192.168.202.143@tcp [ 1913.263082] Key type .llcrypt unregistered [ 1913.265544] Key type ._llcrypt unregistered [ 1937.382769] Key type ._llcrypt registered [ 1937.387810] Key type .llcrypt registered [ 1937.546530] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 1938.822596] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1938.944879] alg: No test for adler32 (adler32-zlib) [ 1940.285615] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 1940.624496] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 1942.442384] Key type lgssc registered [ 1943.872331] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1952.094487] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1952.119484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1953.579849] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1958.076880] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1966.565929] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1973.702801] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1974.139337] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1978.259502] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:195 to 0x240000400:385) [ 1981.035267] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 1988.794317] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1995.488437] Lustre: Failing over lustre-MDT0000 [ 1996.117238] Lustre: server umount lustre-MDT0000 complete [ 2006.232162] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2006.517092] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2006.938734] Lustre: *** cfs_fail_loc=a05, val=0*** [ 2007.116751] LustreError: 62503:0:(obd_config.c:845:class_setup()) setup lustre-MDT0000 failed (-9) [ 2007.120851] LustreError: 62503:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.143@tcp: cfg command failed: rc = -9 [ 2007.127105] Lustre: cmd=cf003 0:lustre-MDT0000 1:lustre-MDT0000_UUID 2:0 3:lustre-MDT0000-mdtlov 4:f [ 2007.127105] [ 2007.134760] LustreError: MGC192.168.202.143@tcp: Confguration from log lustre-MDT0000 failed from MGS -9. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info [ 2007.150370] LustreError: 62475:0:(tgt_mount.c:1884:server_start_targets()) failed to start server lustre-MDT0000: -9 [ 2007.161688] LustreError: 62475:0:(tgt_mount.c:2613:server_fill_super()) Unable to start targets: -9 [ 2007.168960] LustreError: 62475:0:(tgt_mount.c:2128:server_put_super()) no obd lustre-MDT0000 [ 2007.178045] LustreError: 62475:0:(tgt_mount.c:116:server_deregister_mount()) lustre-MDT0000 not registered [ 2007.314354] Lustre: server umount lustre-MDT0000 complete [ 2007.319946] LustreError: 62475:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -9 [ 2014.296814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2014.533872] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2014.543733] LustreError: 60301:0:(mgc_request.c:614:do_requeue()) failed processing log: -5 [ 2014.885754] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2014.928473] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2015.905173] Lustre: 59612:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790781006/real 1790781006] req@000000006c7ffadd x1877769922292864/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790781022 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2015.945570] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2015.985097] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2017.318624] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects [ 2017.352688] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted. [ 2017.393882] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:195 to 0x240000400:417) [ 2019.277690] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2020.712441] Lustre: 59610:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790781011/real 1790781011] req@0000000047c1c0f0 x1877769922293120/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790781027 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2025.440576] Lustre: 59611:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790781016/real 1790781016] req@0000000067c5d177 x1877769922294656/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790781032 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2029.562791] Lustre: server umount lustre-OST0000 complete [ 2043.047731] Lustre: server umount lustre-MDT0000 complete [ 2052.745424] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 2055.284522] Key type lgssc unregistered [ 2055.617423] LNet: 64091:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2055.626705] LNetError: 64091:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2055.643667] LNet: Removed LNI 192.168.202.143@tcp [ 2056.296625] Key type .llcrypt unregistered [ 2056.299933] Key type ._llcrypt unregistered [ 2069.635190] Key type ._llcrypt registered [ 2069.638184] Key type .llcrypt registered [ 2069.755865] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 2070.169299] Key type .llcrypt unregistered [ 2070.170565] Key type ._llcrypt unregistered [ 2097.513778] Key type ._llcrypt registered [ 2097.514830] Key type .llcrypt registered [ 2097.567525] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 2098.284793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2098.369148] alg: No test for adler32 (adler32-zlib) [ 2099.353873] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 2099.550472] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 2101.216313] Key type lgssc registered [ 2102.393053] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2110.177735] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 2110.228913] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2112.257210] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2117.834605] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2127.230569] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2134.981476] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2135.299331] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2137.327590] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:195 to 0x240000400:449) [ 2140.594663] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2148.646072] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2163.170559] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2163.186597] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2163.208390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2164.616566] Lustre: server umount lustre-OST0000 complete [ 2169.587899] Lustre: server umount lustre-MDT0000 complete [ 2182.351899] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 2184.885668] Key type lgssc unregistered [ 2185.171704] LNet: 68426:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2185.191793] LNetError: 68426:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2185.216880] LNet: Removed LNI 192.168.202.143@tcp [ 2186.194364] Key type .llcrypt unregistered [ 2186.196228] Key type ._llcrypt unregistered [ 2213.292126] Key type ._llcrypt registered [ 2213.301776] Key type .llcrypt registered [ 2213.488434] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 2215.742573] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2215.756584] alg: No test for adler32 (adler32-zlib) [ 2216.961234] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 2217.224614] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 2218.857394] Key type lgssc registered [ 2220.269324] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2229.437576] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 2229.457233] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2230.766667] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2235.483575] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2243.605161] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2252.405456] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2252.922583] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2254.968232] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:451 to 0x240000400:481) [ 2261.313209] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2270.596796] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2280.930261] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2280.996795] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2286.050786] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2291.178765] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2292.192231] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2292.421667] Lustre: server umount lustre-OST0000 complete [ 2297.770806] Lustre: server umount lustre-MDT0000 complete [ 2309.577986] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 2312.298937] Key type lgssc unregistered [ 2312.533060] LNet: 72277:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2312.537971] LNetError: 72277:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2312.557893] LNet: Removed LNI 192.168.202.143@tcp [ 2313.340922] Key type .llcrypt unregistered [ 2313.343675] Key type ._llcrypt unregistered [ 2337.993427] Key type ._llcrypt registered [ 2337.995862] Key type .llcrypt registered [ 2338.112151] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 2339.323294] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2339.364817] alg: No test for adler32 (adler32-zlib) [ 2340.508382] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 2340.726108] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 2342.416201] Key type lgssc registered [ 2343.962389] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2352.444737] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 2352.488035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2354.173689] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2358.962939] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2366.366190] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2373.360431] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2373.640670] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2379.763735] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:451 to 0x240000400:513) [ 2380.843980] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2389.517832] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2402.795285] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 2402.806290] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2402.826759] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2405.347693] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2407.367125] Lustre: server umount lustre-OST0000 complete [ 2412.282847] Lustre: server umount lustre-MDT0000 complete [ 2426.067836] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 2429.477673] Key type lgssc unregistered [ 2429.834550] LNet: 76123:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2429.843775] LNetError: 76123:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2429.868400] LNet: Removed LNI 192.168.202.143@tcp [ 2431.209934] Key type .llcrypt unregistered [ 2431.211951] Key type ._llcrypt unregistered [ 2459.480674] Key type ._llcrypt registered [ 2459.485932] Key type .llcrypt registered [ 2459.673472] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 2461.880185] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2461.898505] alg: No test for adler32 (adler32-zlib) [ 2462.925879] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 2463.142118] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 2464.832285] Key type lgssc registered [ 2466.204918] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2474.823995] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 2474.856231] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2490.464780] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2495.584822] LustreError: 77529:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.143@tcp: failed processing log, type 4: rc = -110 [ 2522.272242] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2529.949340] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2538.742561] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2545.589660] Lustre: server umount lustre-OST0000 complete [ 2554.576143] Lustre: DEBUG MARKER: == conf-sanity test 10a: find lctl param broken symlinks ========================================================== 11:19:19 (1790781559) [ 2562.484230] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2562.990510] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2568.823807] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2577.238880] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2585.834402] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2586.321431] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2588.338907] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:516 to 0x240000400:545) [ 2593.462197] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2603.192190] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2619.368294] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2619.373835] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2619.383328] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2622.854391] Lustre: server umount lustre-OST0000 complete [ 2627.966544] Lustre: server umount lustre-MDT0000 complete [ 2641.860779] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 2645.082419] Key type lgssc unregistered [ 2645.414830] LNet: 81732:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2645.420261] LNetError: 81732:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2645.433841] LNet: Removed LNI 192.168.202.143@tcp [ 2646.376883] Key type .llcrypt unregistered [ 2646.378041] Key type ._llcrypt unregistered [ 2693.024616] Key type ._llcrypt registered [ 2693.027438] Key type .llcrypt registered [ 2693.157902] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 2694.199612] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2694.373219] alg: No test for adler32 (adler32-zlib) [ 2695.539866] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 2695.899353] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 2697.640214] Key type lgssc registered [ 2699.438979] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2709.243942] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 2709.275199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2710.714170] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2714.994200] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2724.035655] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2732.321573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2732.771881] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2734.845819] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:516 to 0x240000400:577) [ 2740.130147] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2748.026964] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2760.675229] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2760.698058] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2761.803733] Lustre: server umount lustre-OST0000 complete [ 2767.221168] Lustre: server umount lustre-MDT0000 complete [ 2781.426482] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 2785.782180] Key type lgssc unregistered [ 2786.342752] LNet: 86356:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2786.356737] LNetError: 86356:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2787.430829] LNet: Removed LNI 192.168.202.143@tcp [ 2788.437377] Key type .llcrypt unregistered [ 2788.440958] Key type ._llcrypt unregistered [ 2812.127282] Key type ._llcrypt registered [ 2812.129150] Key type .llcrypt registered [ 2812.331845] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 2813.787622] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2813.824469] alg: No test for adler32 (adler32-zlib) [ 2815.292425] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 2815.655187] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 2817.328253] Key type lgssc registered [ 2818.695831] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2825.545782] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 2825.592580] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2841.120424] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2846.241209] LustreError: 87611:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.143@tcp: failed processing log, type 4: rc = -110 [ 2872.864544] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2880.102857] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2888.655959] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2895.472731] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2895.660975] LustreError: MGC192.168.202.143@tcp: Confguration from log lustre-MDT0000 failed from MGS -2. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info [ 2895.669303] LustreError: 88633:0:(tgt_mount.c:1884:server_start_targets()) failed to start server lustre-MDT0000: -2 [ 2895.673894] LustreError: 88633:0:(tgt_mount.c:2613:server_fill_super()) Unable to start targets: -2 [ 2895.678778] LustreError: 88633:0:(tgt_mount.c:2128:server_put_super()) no obd lustre-MDT0000 [ 2895.741340] Lustre: server umount lustre-MDT0000 complete [ 2895.743434] LustreError: 88633:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -2 [ 2915.808464] Lustre: 86938:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790781907/real 1790781907] req@00000000d91c3eca x1877770839788544/t0(0) o400->MGC192.168.202.143@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1790781923 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2915.846248] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2915.863736] LustreError: 87623:0:(mgc_request.c:614:do_requeue()) failed processing log: -5 [ 2916.029929] Lustre: server umount lustre-OST0000 complete [ 2926.014611] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 2935.988595] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 2966.972159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2967.282607] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2967.335486] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2967.425318] Lustre: lustre-MDT0000: new disk, initializing [ 2967.529177] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2967.558194] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2973.481641] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 2983.948566] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2991.825045] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2992.234851] Lustre: lustre-OST0000: new disk, initializing [ 2992.245092] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2992.254194] Lustre: Skipped 1 previous similar message [ 2992.259234] Lustre: 92012:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2992.419871] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2999.439253] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2999.456123] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2999.506849] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3000.052670] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3010.459685] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3015.459860] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3015.678950] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3019.753142] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3019.782290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3023.656751] Lustre: server umount lustre-OST0000 complete [ 3028.255681] Lustre: server umount lustre-MDT0000 complete [ 3038.473615] Lustre: DEBUG MARKER: == conf-sanity test 18: check mkfs creates large journals ========================================================== 11:27:23 (1790782043) [ 3042.370412] Lustre: DEBUG MARKER: use device /dev/mapper/mds1_flakey with MIN=2000000 [ 3059.220075] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 3070.751448] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 3103.113864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3103.419231] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3103.463511] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3103.601386] Lustre: lustre-MDT0000: new disk, initializing [ 3103.689894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3103.736761] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3109.474729] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3119.932723] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3127.751671] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3128.149591] Lustre: lustre-OST0000: new disk, initializing [ 3128.153781] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3128.168181] Lustre: Skipped 1 previous similar message [ 3128.181576] Lustre: 97024:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3128.287423] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3135.000558] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3137.147442] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3137.162305] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3137.243028] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3146.619107] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3151.775137] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3151.995700] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3157.478310] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3157.499738] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3159.905297] Lustre: server umount lustre-OST0000 complete [ 3163.463456] Lustre: server umount lustre-MDT0000 complete [ 3171.852543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3172.312421] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3177.518728] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3188.997291] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3197.478054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3198.218609] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3206.490880] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3214.764829] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3228.686619] Lustre: DEBUG MARKER: Success: mkfs creates large journals. Size: 78M [ 3238.371291] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3238.393583] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3239.998365] Lustre: server umount lustre-OST0000 complete [ 3244.056848] Lustre: server umount lustre-MDT0000 complete [ 3257.120418] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 3261.634405] Key type lgssc unregistered [ 3262.146503] LNet: 101310:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3262.155547] LNetError: 101310:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3262.197818] LNet: Removed LNI 192.168.202.143@tcp [ 3263.727179] Key type .llcrypt unregistered [ 3263.739487] Key type ._llcrypt unregistered [ 3280.716347] Key type ._llcrypt registered [ 3280.720726] Key type .llcrypt registered [ 3280.896696] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 3296.285688] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 3297.367770] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3297.465173] alg: No test for adler32 (adler32-zlib) [ 3298.704717] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 3298.957656] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 3300.648235] Key type lgssc registered [ 3302.203218] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3329.403054] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 3329.433654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3330.715276] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3330.752601] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3330.865556] Lustre: lustre-MDT0000: new disk, initializing [ 3330.946953] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3330.977825] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3335.056954] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3344.502534] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3351.654903] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3351.995591] Lustre: lustre-OST0000: new disk, initializing [ 3352.006064] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3352.012838] Lustre: Skipped 1 previous similar message [ 3352.018456] Lustre: 105062:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3352.130443] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3358.066863] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3358.829600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3358.844892] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3358.867881] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3367.697218] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3372.469922] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3372.734291] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3379.171375] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3379.178976] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3379.193097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3384.289546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3389.411625] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3389.426171] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3389.616060] Lustre: server umount lustre-OST0000 complete [ 3393.779458] Lustre: server umount lustre-MDT0000 complete [ 3403.561642] Lustre: DEBUG MARKER: == conf-sanity test 19a: start/stop MDS without OSTs ===== 11:33:28 (1790782408) [ 3411.629775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3412.241674] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3417.520873] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3424.827449] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3427.476537] Lustre: server umount lustre-MDT0000 complete [ 3437.037112] Lustre: DEBUG MARKER: == conf-sanity test 19b: start/stop OSTs without MDS ===== 11:34:02 (1790782442) [ 3444.431912] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3460.000669] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3465.121217] LustreError: 108306:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.143@tcp: failed processing log, type 4: rc = -110 [ 3490.848461] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3497.218939] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3505.231223] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3507.626222] Lustre: server umount lustre-OST0000 complete [ 3516.470728] Lustre: DEBUG MARKER: == conf-sanity test 20: remount ro,rw mounts work and doesn't break /etc/mtab ========================================================== 11:35:22 (1790782522) [ 3523.971827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3524.514803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3529.316477] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3537.377261] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3544.009323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3544.382579] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3551.307607] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3559.087230] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3570.146877] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3570.165978] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3573.187517] Lustre: server umount lustre-MDT0000 complete [ 3576.495908] LustreError: 109729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790782583 with bad export cookie 15845700808941560787 [ 3576.501515] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3576.508395] LustreError: 109729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3576.694308] Lustre: server umount lustre-OST0000 complete [ 3584.959988] Lustre: DEBUG MARKER: == conf-sanity test 21a: start mds before ost, stop ost first ========================================================== 11:36:30 (1790782590) [ 3592.511106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3593.013842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3597.593734] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3604.902904] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3611.714951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3618.301176] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:36 to 0x240000400:65) [ 3618.680346] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3626.963408] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3632.688760] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3632.883323] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3638.755875] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3638.757040] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3638.787057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3641.407621] Lustre: server umount lustre-OST0000 complete [ 3645.996058] Lustre: server umount lustre-MDT0000 complete [ 3654.555665] Lustre: DEBUG MARKER: == conf-sanity test 21b: start ost before mds, stop mds first ========================================================== 11:37:40 (1790782660) [ 3661.675939] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3677.347581] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3682.465301] LustreError: 115071:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.143@tcp: failed processing log, type 4: rc = -110 [ 3708.194732] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3708.211694] Lustre: Skipped 1 previous similar message [ 3715.678701] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3724.007187] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3731.210432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3731.678697] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:36 to 0x240000400:97) [ 3736.282921] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3743.416337] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3747.290596] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3747.568753] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3749.545170] Lustre: server umount lustre-MDT0000 complete [ 3753.353779] LustreError: 115079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790782760 with bad export cookie 15845700808941562243 [ 3753.362870] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3763.874750] Lustre: server umount lustre-OST0000 complete [ 3772.435637] Lustre: DEBUG MARKER: == conf-sanity test 21c: start mds between two osts, stop mds last ========================================================== 11:39:37 (1790782777) [ 3779.135555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3794.657850] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3799.840887] LustreError: 117889:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.143@tcp: failed processing log, type 4: rc = -110 [ 3825.504920] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3825.515873] Lustre: Skipped 1 previous similar message [ 3831.858844] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3838.568864] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3845.055035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3845.627482] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:36 to 0x240000400:129) [ 3850.015386] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3856.811477] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3863.737230] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3863.883252] Lustre: lustre-OST0001: new disk, initializing [ 3863.887238] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3863.901276] Lustre: 119848:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3868.542585] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3871.261528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3871.267515] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3871.278729] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3878.524267] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3883.316928] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3883.548471] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3886.564031] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3886.590487] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3899.872181] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3899.909336] Lustre: server umount lustre-OST0000 complete [ 3907.044051] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3907.046882] LustreError: 117905:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3907.053701] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3907.067032] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3907.067040] Lustre: Skipped 2 previous similar messages [ 3907.096647] LustreError: 117905:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3912.161202] LustreError: 117906:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3917.284958] LustreError: 117905:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3918.304134] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3918.489395] Lustre: server umount lustre-OST0001 complete [ 3922.717387] Lustre: server umount lustre-MDT0000 complete [ 3943.265610] Lustre: DEBUG MARKER: == conf-sanity test 21d: start mgs then ost and then mds ========================================================== 11:42:28 (1790782948) [ 3945.139182] Lustre: DEBUG MARKER: SKIP: conf-sanity test_21d need separate mgs device [ 3946.789153] Lustre: DEBUG MARKER: SKIP: conf-sanity test_21e skipping excluded test 21e [ 3948.599976] Lustre: DEBUG MARKER: == conf-sanity test 22: start a client before osts (should return errs) ========================================================== 11:42:34 (1790782954) [ 3955.622747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3956.009848] Lustre: MGS: Logs for fs lustre were removed by user request. All servers must be restarted in order to regenerate the logs: rc = 0 [ 3956.059631] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3956.290315] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3956.295462] Lustre: Skipped 2 previous similar messages [ 3960.813460] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3969.063747] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3975.757935] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3975.981693] Lustre: MGS: Regenerating lustre-OST0000 log by user request: rc = 0 [ 3982.902275] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 3984.985955] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:36 to 0x240000400:161) [ 3990.624787] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3994.532930] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3994.804182] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4000.232111] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4000.239130] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4000.252395] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4000.254439] Lustre: Skipped 2 previous similar messages [ 4002.873914] Lustre: server umount lustre-OST0000 complete [ 4026.351206] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4032.362599] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4039.947816] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4039.977905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.996552] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4044.658206] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4045.055193] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4051.528924] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ed9a0675800.ost_server_uuid 50 [ 4053.376370] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8ed9a0675800.ost_server_uuid in FULL state after 0 sec [ 4053.692579] LustreError: 122741:0:(ldlm_lockd.c:727:ldlm_handle_ast_error()) ### client (nid 192.168.202.43@tcp) returned error from blocking AST (req@0000000035ec9e9b x1877771347403008 status -107 rc -107), evict it ns: mdt-lustre-MDT0000_UUID lock: 00000000c5efbcb1/0xdbe73cee7e14f266 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.43@tcp remote: 0xacd08213b0c7117 expref: 5 pid: 122743 timeout: 4153 lvb_type: 0 lru_score: 0 lru_type: 0 [ 4053.718142] LustreError: lustre-MDT0000: A client on nid 192.168.202.43@tcp was evicted due to a lock blocking callback time out: rc -107 [ 4053.722184] LustreError: 122731:0:(ldlm_lockd.c:255:expired_lock_main()) ### lock callback timer expired after 0s: evicting client at 192.168.202.43@tcp ns: mdt-lustre-MDT0000_UUID lock: 00000000c5efbcb1/0xdbe73cee7e14f266 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.43@tcp remote: 0xacd08213b0c7117 expref: 6 pid: 122743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 [ 4060.131663] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4060.146360] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4064.784239] Lustre: server umount lustre-OST0000 complete [ 4068.907656] Lustre: server umount lustre-MDT0000 complete [ 4081.311390] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4084.714317] Key type lgssc unregistered [ 4084.984192] LNet: 127184:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4085.000517] LNetError: 127184:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4085.025825] LNet: Removed LNI 192.168.202.143@tcp [ 4085.954940] Key type .llcrypt unregistered [ 4085.958115] Key type ._llcrypt unregistered [ 4112.463986] Key type ._llcrypt registered [ 4112.466703] Key type .llcrypt registered [ 4112.599596] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4113.900705] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4114.156897] alg: No test for adler32 (adler32-zlib) [ 4115.202359] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4115.438326] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4117.112315] Key type lgssc registered [ 4118.364894] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4126.016476] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4126.034471] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4127.577507] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4131.796201] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4140.087729] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4147.087069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4147.408679] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4152.557162] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:163 to 0x240000400:193) [ 4153.591378] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4162.120254] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4168.475978] Lustre: Failing over lustre-MDT0000 [ 4169.065583] Lustre: server umount lustre-MDT0000 complete [ 4182.215073] LustreError: 128591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 192.168.202.43@tcp arrived at 1790783189 with bad export cookie 6467610330688739173 [ 4189.600746] Lustre: 127910:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790783180/real 1790783180] req@000000009bad73aa x1877772202945792/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790783196 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4189.640860] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4189.665367] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4194.596414] Lustre: 127912:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790783185/real 1790783185] req@000000006fb915e6 x1877772202946048/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790783201 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4194.619835] Lustre: 127912:0:(client.c:2504:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 4198.883552] Lustre: 127911:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790783190/real 1790783190] req@000000004ecf63f4 x1877772202946304/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790783206 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4199.682483] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4203.937400] Lustre: 127913:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790783195/real 1790783195] req@0000000008ad2b6c x1877772202946560/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790783211 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4214.261698] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x59c19154325957c0 [ 4214.281835] Lustre: MGC192.168.202.143@tcp: Connection restored to 0@lo (at 0@lo) [ 4214.895780] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4214.941863] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 4220.151927] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4222.302602] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects [ 4222.307271] Lustre: lustre-MDT0000: Denying connection for new client 39aa9b23-5c18-4ba8-a819-32ef77209a6e (at 192.168.202.43@tcp), waiting for 1 known clients (0 recovered, 0 in progress, and 0 evicted) to recover in 1:00 [ 4235.245592] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4235.266759] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4237.105538] Lustre: server umount lustre-OST0000 complete [ 4240.860472] LustreError: 131444:0:(ldlm_lib.c:3038:target_stop_recovery_thread()) lustre-MDT0000: Aborting recovery [ 4240.877660] Lustre: 130797:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 4240.888977] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 4240.919494] Lustre: lustre-MDT0000: Recovery over after 0:18, of 1 clients 0 recovered and 1 was evicted. [ 4241.504754] Lustre: server umount lustre-MDT0000 complete [ 4253.638838] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4256.982651] Key type lgssc unregistered [ 4257.285990] LNet: 131994:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4257.301084] LNetError: 131994:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4257.323231] LNet: Removed LNI 192.168.202.143@tcp [ 4258.420243] Key type .llcrypt unregistered [ 4258.423071] Key type ._llcrypt unregistered [ 4285.458287] Key type ._llcrypt registered [ 4285.460457] Key type .llcrypt registered [ 4285.607385] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4286.683964] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4286.698422] alg: No test for adler32 (adler32-zlib) [ 4287.682666] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4287.884233] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4289.552917] Key type lgssc registered [ 4290.779462] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4299.016638] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4299.030271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4300.717858] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4305.873139] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4314.423698] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4321.780314] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4322.155711] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4325.259298] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:163 to 0x240000400:225) [ 4328.655525] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4336.068643] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4344.502955] Lustre: server umount lustre-OST0000 complete [ 4358.369152] Lustre: server umount lustre-MDT0000 complete [ 4371.707684] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4374.595516] Key type lgssc unregistered [ 4375.002863] LNet: 135844:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4375.021507] LNetError: 135844:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4376.046049] LNet: Removed LNI 192.168.202.143@tcp [ 4377.126864] Key type .llcrypt unregistered [ 4377.130627] Key type ._llcrypt unregistered [ 4408.587784] Key type ._llcrypt registered [ 4408.589499] Key type .llcrypt registered [ 4408.672982] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4409.814942] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4409.839972] alg: No test for adler32 (adler32-zlib) [ 4410.918507] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4411.247874] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4412.920936] Key type lgssc registered [ 4414.131179] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4421.756383] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4421.806856] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4423.366063] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4428.213171] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4434.544617] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4440.809447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4441.059376] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4448.256540] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:163 to 0x240000400:257) [ 4448.734037] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4458.137938] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4468.706701] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4468.706946] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4468.725554] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4473.353930] Lustre: server umount lustre-OST0000 complete [ 4477.436446] Lustre: server umount lustre-MDT0000 complete [ 4488.906827] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4491.780746] Key type lgssc unregistered [ 4492.050792] LNet: 139982:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4492.058544] LNetError: 139982:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4492.083757] LNet: Removed LNI 192.168.202.143@tcp [ 4493.067703] Key type .llcrypt unregistered [ 4493.069383] Key type ._llcrypt unregistered [ 4514.569330] Key type ._llcrypt registered [ 4514.572858] Key type .llcrypt registered [ 4514.714804] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4515.959960] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4516.000297] alg: No test for adler32 (adler32-zlib) [ 4517.116338] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4517.459269] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4519.184257] Key type lgssc registered [ 4520.309581] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4531.038884] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4531.076268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4532.483956] Lustre: *** cfs_fail_loc=135, val=0*** [ 4532.565026] LustreError: 141513:0:(obd_config.c:845:class_setup()) setup lustre-MDT0000 failed (-2) [ 4532.579469] LustreError: 141513:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.143@tcp: cfg command failed: rc = -2 [ 4532.591805] Lustre: cmd=cf003 0:lustre-MDT0000 1:lustre-MDT0000_UUID 2:0 3:lustre-MDT0000-mdtlov 4:f [ 4532.591805] [ 4532.604830] LustreError: MGC192.168.202.143@tcp: Confguration from log lustre-MDT0000 failed from MGS -2. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info [ 4532.614099] LustreError: 141473:0:(tgt_mount.c:1884:server_start_targets()) failed to start server lustre-MDT0000: -2 [ 4532.621531] LustreError: 141473:0:(tgt_mount.c:2613:server_fill_super()) Unable to start targets: -2 [ 4532.630646] LustreError: 141473:0:(tgt_mount.c:2128:server_put_super()) no obd lustre-MDT0000 [ 4532.638389] LustreError: 141473:0:(tgt_mount.c:116:server_deregister_mount()) lustre-MDT0000 not registered [ 4534.761766] Lustre: server umount lustre-MDT0000 complete [ 4534.765660] LustreError: 141473:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -2 [ 4541.341201] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4541.735678] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4545.545420] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4552.474112] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4554.119985] Lustre: server umount lustre-MDT0000 complete [ 4567.210515] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4572.078145] Key type lgssc unregistered [ 4572.455952] LNet: 143093:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4572.469854] LNetError: 143093:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4572.505580] LNet: Removed LNI 192.168.202.143@tcp [ 4573.556845] Key type .llcrypt unregistered [ 4573.565166] Key type ._llcrypt unregistered [ 4595.444761] Key type ._llcrypt registered [ 4595.447075] Key type .llcrypt registered [ 4595.537983] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4595.853772] Key type .llcrypt unregistered [ 4595.855574] Key type ._llcrypt unregistered [ 4613.390873] Key type ._llcrypt registered [ 4613.394256] Key type .llcrypt registered [ 4613.503032] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4614.753356] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4614.788819] alg: No test for adler32 (adler32-zlib) [ 4615.864876] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4616.054548] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4617.720225] Key type lgssc registered [ 4619.105321] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4627.391507] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4627.404851] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4642.977797] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4648.096702] LustreError: 145088:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.143@tcp: failed processing log, type 4: rc = -110 [ 4674.848232] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4682.207725] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4688.857877] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4695.544849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4696.505661] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4696.591933] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:259 to 0x240000400:289) [ 4702.575395] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4713.326414] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4716.698699] Lustre: Setting parameter lustre-OST0000.ost.client_cache_seconds=115 in log lustre-OST0000 [ 4727.266206] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4727.283713] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4727.309308] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4729.314540] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4731.459230] Lustre: server umount lustre-OST0000 complete [ 4736.491223] Lustre: server umount lustre-MDT0000 complete [ 4747.460204] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4750.830701] Key type lgssc unregistered [ 4751.372336] LNet: 147891:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4751.391549] LNetError: 147891:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4751.416378] LNet: Removed LNI 192.168.202.143@tcp [ 4752.670585] Key type .llcrypt unregistered [ 4752.673643] Key type ._llcrypt unregistered [ 4774.315399] Key type ._llcrypt registered [ 4774.318297] Key type .llcrypt registered [ 4774.409906] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4775.795029] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4775.827083] alg: No test for adler32 (adler32-zlib) [ 4777.003817] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4777.295295] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4778.992488] Key type lgssc registered [ 4780.038744] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4786.841862] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4786.861972] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4788.261417] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4792.362838] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4799.786502] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4806.629056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4807.146339] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4813.294548] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:259 to 0x240000400:321) [ 4813.375229] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4821.250967] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4825.577553] Lustre: Failing over lustre-MDT0000 [ 4825.917619] Lustre: server umount lustre-MDT0000 complete [ 4843.808102] Lustre: 148599:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790783835/real 1790783835] req@00000000ef632807 x1877772897102464/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790783851 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4843.837750] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4843.874665] LustreError: MGC192.168.202.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4844.531379] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4849.952598] Lustre: 148597:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790783841/real 1790783841] req@000000007c8757b1 x1877772897102720/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790783857 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4849.991667] Lustre: 148597:0:(client.c:2504:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 4854.241664] LustreError: 148596:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000047531658 x1877772897103488/t0(0) o250->MGC192.168.202.143@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4854.861506] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4854.920983] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 4855.904200] Lustre: 148600:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790783846/real 1790783846] req@000000007c71c0d5 x1877772897102976/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790783862 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4860.638845] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4865.257191] Lustre: Setting parameter lustre-MDT0000.mdt.identity_acquire_expire=35 in log lustre-MDT0000 [ 4869.092806] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 4869.292500] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects [ 4869.346290] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted. [ 4869.406386] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:259 to 0x240000400:353) [ 4873.027169] Lustre: Setting parameter lustre-MDT0000-mdc.mdc.max_rpcs_in_flight=13 in log lustre-client [ 4889.570633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4889.587742] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4893.523887] Lustre: server umount lustre-OST0000 complete [ 4897.791242] Lustre: server umount lustre-MDT0000 complete [ 4909.607201] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 4912.368405] Key type lgssc unregistered [ 4912.687930] LNet: 153099:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4912.701827] LNetError: 153099:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4912.720940] LNet: Removed LNI 192.168.202.143@tcp [ 4913.542434] Key type .llcrypt unregistered [ 4913.545192] Key type ._llcrypt unregistered [ 4936.139874] Key type ._llcrypt registered [ 4936.143421] Key type .llcrypt registered [ 4936.237986] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 4937.374264] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4937.387287] alg: No test for adler32 (adler32-zlib) [ 4938.421745] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 4938.619201] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 4940.354799] Key type lgssc registered [ 4941.476186] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4948.059960] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4948.075637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4949.597692] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4954.184354] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4964.116228] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4970.946456] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4971.269539] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4974.326511] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:355 to 0x240000400:385) [ 4978.119521] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 4985.132487] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4989.069898] Lustre: Setting parameter lustre-client.llite.max_read_ahead_whole_mb=5 in log lustre-client [ 4992.332581] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=6 in log lustre-client [ 5007.373385] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=4 in log lustre-client [ 5020.646475] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5020.652740] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5020.672098] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5025.313609] Lustre: server umount lustre-OST0000 complete [ 5029.374717] Lustre: server umount lustre-MDT0000 complete [ 5040.070203] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 5042.907739] Key type lgssc unregistered [ 5043.200300] LNet: 157255:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5043.204752] LNetError: 157255:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5044.262664] LNet: Removed LNI 192.168.202.143@tcp [ 5045.044376] Key type .llcrypt unregistered [ 5045.048208] Key type ._llcrypt unregistered [ 5062.227243] Key type ._llcrypt registered [ 5062.228922] Key type .llcrypt registered [ 5062.325822] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 5063.703380] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5063.797130] alg: No test for adler32 (adler32-zlib) [ 5064.853141] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 5065.060991] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 5066.795777] Key type lgssc registered [ 5068.041584] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5078.556023] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5078.598825] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5080.172493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5086.581543] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5095.553860] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5102.691795] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5103.224396] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5105.275263] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:355 to 0x240000400:417) [ 5110.600092] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5118.880061] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5124.436313] Lustre: Modifying parameter lustre-OST0000.ost.client_cache_seconds=230 in log lustre-OST0000 [ 5138.081695] Lustre: Modifying parameter lustre-OST0000.ost.client_cache_seconds=115 in log lustre-OST0000 [ 5152.304417] Lustre: Setting parameter lustre-OST0000.osd.auto_scrub=1 in log lustre-OST0000 [ 5159.848456] Lustre: Modifying parameter lustre-OST0000.osd.auto_scrub=2592000 in log lustre-OST0000 [ 5167.077312] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5167.089114] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5167.104225] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5172.194430] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5172.837573] Lustre: server umount lustre-OST0000 complete [ 5177.193680] Lustre: server umount lustre-MDT0000 complete [ 5189.416026] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 5193.139441] Key type lgssc unregistered [ 5193.613837] LNet: 162472:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5193.625671] LNetError: 162472:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5193.647417] LNet: Removed LNI 192.168.202.143@tcp [ 5194.980977] Key type .llcrypt unregistered [ 5194.984601] Key type ._llcrypt unregistered [ 5221.475200] Key type ._llcrypt registered [ 5221.479137] Key type .llcrypt registered [ 5221.713819] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 5222.949856] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5222.993048] alg: No test for adler32 (adler32-zlib) [ 5224.036749] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 5224.221500] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 5225.856368] Key type lgssc registered [ 5227.745135] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5239.454120] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5239.543989] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5241.579576] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5247.519649] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5257.365746] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5266.167475] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5266.726044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5272.044267] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:355 to 0x240000400:449) [ 5273.704501] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5282.379450] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5302.762652] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5302.764759] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5302.789974] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5306.080667] Lustre: server umount lustre-OST0000 complete [ 5310.805615] Lustre: server umount lustre-MDT0000 complete [ 5324.654665] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 5327.984489] Key type lgssc unregistered [ 5328.418816] LNet: 166470:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5328.437045] LNetError: 166470:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5328.459954] LNet: Removed LNI 192.168.202.143@tcp [ 5329.406216] Key type .llcrypt unregistered [ 5329.408746] Key type ._llcrypt unregistered [ 5351.356633] Key type ._llcrypt registered [ 5351.360423] Key type .llcrypt registered [ 5351.570230] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 5353.216528] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5353.226482] alg: No test for adler32 (adler32-zlib) [ 5354.423862] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 5354.651734] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 5356.416269] Key type lgssc registered [ 5357.537299] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5367.560880] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5367.576577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5369.471102] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5377.127488] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5385.616289] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5393.143867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5393.440420] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5394.504080] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:355 to 0x240000400:481) [ 5401.480516] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5410.854632] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5430.248770] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5430.249812] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5430.250389] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5434.618320] Lustre: server umount lustre-OST0000 complete [ 5439.254038] Lustre: server umount lustre-MDT0000 complete [ 5452.575064] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 5456.464478] Key type lgssc unregistered [ 5456.885455] LNet: 170332:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5456.896434] LNetError: 170332:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5456.928937] LNet: Removed LNI 192.168.202.143@tcp [ 5458.187469] Key type .llcrypt unregistered [ 5458.191981] Key type ._llcrypt unregistered [ 5482.405483] Key type ._llcrypt registered [ 5482.410726] Key type .llcrypt registered [ 5482.627701] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 5484.051390] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5484.266387] alg: No test for adler32 (adler32-zlib) [ 5485.422614] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 5485.697497] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 5487.464215] Key type lgssc registered [ 5488.978566] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5497.558141] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5497.572323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5498.887995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5504.494744] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5512.655691] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5519.761886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5520.268968] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5524.416563] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:355 to 0x240000400:513) [ 5528.393600] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5537.781379] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5550.013690] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5550.111026] Lustre: MGS: Regenerating lustre-OST0001 log by user request: rc = 0 [ 5550.338048] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5557.147996] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5567.537166] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5569.429703] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 5581.110752] Lustre: Permanently deactivating lustre-OST0001 [ 5581.132166] Lustre: Setting parameter lustre-OST0001-osc.osc.active=0 in log lustre-client [ 5587.627264] Lustre: setting import lustre-OST0001_UUID INACTIVE by administrator request [ 5591.441566] Lustre: Permanently reactivating lustre-OST0001 [ 5591.468737] Lustre: Modifying parameter lustre-OST0001-osc.osc.active=1 in log lustre-client [ 5591.482328] Lustre: Skipped 1 previous similar message [ 5593.431233] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5593.460387] Lustre: lustre-OST0001: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5593.484958] LustreError: lustre-OST0001-osc-MDT0000: This client was evicted by lustre-OST0001; in progress operations using this service will fail. [ 5593.524048] Lustre: lustre-OST0001-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5608.938544] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5608.967422] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5609.017424] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5612.797866] Lustre: server umount lustre-OST0001 complete [ 5619.690583] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5619.698122] LustreError: 173935:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5619.702013] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5619.702021] Lustre: Skipped 1 previous similar message [ 5619.726830] LustreError: 173935:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5623.917312] Lustre: server umount lustre-OST0000 complete [ 5629.326934] Lustre: server umount lustre-MDT0000 complete [ 5643.295911] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 5646.286102] Key type lgssc unregistered [ 5646.661774] LNet: 175509:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5646.672822] LNetError: 175509:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5646.693412] LNet: Removed LNI 192.168.202.143@tcp [ 5647.915650] Key type .llcrypt unregistered [ 5647.917640] Key type ._llcrypt unregistered [ 5685.320276] Key type ._llcrypt registered [ 5685.323701] Key type .llcrypt registered [ 5685.417546] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 5686.656847] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5686.968045] alg: No test for adler32 (adler32-zlib) [ 5688.196848] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 5688.619478] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 5690.320247] Key type lgssc registered [ 5691.707640] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5700.220418] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5700.245254] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5701.609163] Lustre: MGS: Logs for fs lustre were removed by user request. All servers must be restarted in order to regenerate the logs: rc = 0 [ 5701.634434] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5701.828807] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5707.061798] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5714.837119] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5722.184544] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5722.376584] Lustre: MGS: Regenerating lustre-OST0000 log by user request: rc = 0 [ 5722.590429] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5729.395674] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:355 to 0x240000400:545) [ 5730.319484] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5739.440608] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5743.277320] Lustre: Setting parameter lustre-client.llite.max_read_ahead_whole_mb=1 in log lustre-client [ 5751.258091] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=2 in log lustre-client [ 5758.057824] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=3 in log lustre-client [ 5768.288223] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=4 in log lustre-client [ 5782.423324] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=4 in log lustre-client [ 5782.433901] Lustre: Skipped 1 previous similar message [ 5804.492350] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=1 in log lustre-client [ 5804.508121] Lustre: Skipped 2 previous similar messages [ 5838.801890] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_whole_mb=4 in log lustre-client [ 5838.808908] Lustre: Skipped 4 previous similar messages [ 5913.569982] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5913.578801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5915.966854] Lustre: server umount lustre-OST0000 complete [ 5919.996485] Lustre: server umount lustre-MDT0000 complete [ 5932.469582] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 5935.398768] Key type lgssc unregistered [ 5935.763717] LNet: 181153:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5935.775001] LNetError: 181153:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5935.790377] LNet: Removed LNI 192.168.202.143@tcp [ 5936.810486] Key type .llcrypt unregistered [ 5936.812117] Key type ._llcrypt unregistered [ 5961.897850] Key type ._llcrypt registered [ 5961.911210] Key type .llcrypt registered [ 5962.082450] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 5963.377865] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5963.406697] alg: No test for adler32 (adler32-zlib) [ 5964.599229] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 5964.910499] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 5966.704325] Key type lgssc registered [ 5968.287884] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5977.293877] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5977.329370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5979.062717] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5983.836200] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 5993.095245] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6000.030307] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6000.425702] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6004.547110] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:355 to 0x240000400:577) [ 6008.935893] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_default_debug -1 all [ 6019.254883] Lustre: DEBUG MARKER: oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6039.305568] Lustre: server umount lustre-OST0000 complete [ 6053.626218] Lustre: server umount lustre-MDT0000 complete [ 6065.291515] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 6068.764934] Key type lgssc unregistered [ 6069.196149] LNet: 185158:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6069.202108] LNetError: 185158:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6069.213831] LNet: Removed LNI 192.168.202.143@tcp [ 6070.262718] Key type .llcrypt unregistered [ 6070.267072] Key type ._llcrypt unregistered [ 6095.503870] Key type ._llcrypt registered [ 6095.506958] Key type .llcrypt registered [ 6095.630555] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 6096.364338] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6096.402206] alg: No test for adler32 (adler32-zlib) [ 6097.648431] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 6098.151794] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 6100.008222] Key type lgssc registered [ 6102.228646] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6373.162944] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 6376.834634] Key type lgssc unregistered [ 6377.100715] LNet: 187041:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6377.106155] LNetError: 187041:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6377.131466] LNet: Removed LNI 192.168.202.143@tcp [ 6378.179705] Key type .llcrypt unregistered [ 6378.181403] Key type ._llcrypt unregistered [ 6404.255408] Key type ._llcrypt registered [ 6404.258421] Key type .llcrypt registered [ 6404.382912] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing set_hostid [ 6423.466487] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing load_modules_local [ 6425.177915] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6425.266387] alg: No test for adler32 (adler32-zlib) [ 6426.394256] Lustre: Lustre: Build Version: 2.17.59_2_g696972c [ 6426.832292] LNet: Added LNI 192.168.202.143@tcp [8/256/0/180] [ 6428.584277] Key type lgssc registered [ 6430.084956] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6456.351777] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6201 sec ========== 12:24:21 (1790785461) [ 6458.484907] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 12:24:23 (1790785463) === [ 6462.216292] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 12:24:27 (1790785467) === [ 6496.169194] Lustre: DEBUG MARKER: oleg243-server.virtnet: executing unload_modules_local [ 6500.561375] Key type lgssc unregistered [ 6500.842627] LNet: 191476:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6500.853321] LNetError: 191476:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6500.871698] LNet: Removed LNI 192.168.202.143@tcp [ 6502.107784] Key type .llcrypt unregistered [ 6502.112371] Key type ._llcrypt unregistered