[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 480652194 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] MTRR default type: write-back [ 0.000000] MTRR fixed ranges enabled: [ 0.000000] 00000-9FFFF write-back [ 0.000000] A0000-BFFFF uncachable [ 0.000000] C0000-FFFFF write-protect [ 0.000000] MTRR variable ranges enabled: [ 0.000000] 0 base 0000C0000000 mask 3FFFC0000000 uncachable [ 0.000000] 1 disabled [ 0.000000] 2 disabled [ 0.000000] 3 disabled [ 0.000000] 4 disabled [ 0.000000] 5 disabled [ 0.000000] 6 disabled [ 0.000000] 7 disabled [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] BRK [0x0ce01000, 0x0ce01fff] PGTABLE [ 0.000000] BRK [0x0ce02000, 0x0ce02fff] PGTABLE [ 0.000000] BRK [0x0ce03000, 0x0ce03fff] PGTABLE [ 0.000000] BRK [0x0ce04000, 0x0ce04fff] PGTABLE [ 0.000000] BRK [0x0ce05000, 0x0ce05fff] PGTABLE [ 0.000000] BRK [0x0ce06000, 0x0ce06fff] PGTABLE [ 0.000000] BRK [0x0ce07000, 0x0ce07fff] PGTABLE [ 0.000000] BRK [0x0ce08000, 0x0ce08fff] PGTABLE [ 0.000000] BRK [0x0ce09000, 0x0ce09fff] PGTABLE [ 0.000000] BRK [0x0ce0a000, 0x0ce0afff] PGTABLE [ 0.000000] BRK [0x0ce0b000, 0x0ce0bfff] PGTABLE [ 0.000000] BRK [0x0ce0c000, 0x0ce0cfff] PGTABLE [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] On node 0 totalpages: 1076588 [ 0.000000] DMA zone: 64 pages used for memmap [ 0.000000] DMA zone: 158 pages reserved [ 0.000000] DMA zone: 3998 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 12224 pages used for memmap [ 0.000000] DMA32 zone: 782286 pages, LIFO batch:63 [ 0.000000] Normal zone: 4536 pages used for memmap [ 0.000000] Normal zone: 290304 pages, LIFO batch:63 [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] ACPI: IRQ0 used by override. [ 0.000000] ACPI: IRQ5 used by override. [ 0.000000] ACPI: IRQ9 used by override. [ 0.000000] ACPI: IRQ10 used by override. [ 0.000000] ACPI: IRQ11 used by override. [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] pcpu-alloc: s221184 r8192 d28672 u524288 alloc=1*2097152 [ 0.000000] pcpu-alloc: [0] 0 1 2 3 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001000] hpet clockevent registered [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.003286] x2apic enabled [ 0.004012] Switched APIC routing to physical x2apic. [ 0.005019] kvm-guest: setup PV IPIs [ 0.007383] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008028] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009025] pid_max: default: 32768 minimum: 301 [ 0.010203] LSM: Security Framework initializing [ 0.012112] Yama: becoming mindful. [ 0.013089] SELinux: Initializing. [ 0.014113] *** VALIDATE selinux *** [ 0.024086] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.030020] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.031228] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032149] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.034156] *** VALIDATE tmpfs *** [ 0.036273] *** VALIDATE proc *** [ 0.038167] *** VALIDATE cgroup *** [ 0.039021] *** VALIDATE cgroup2 *** [ 0.040376] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.042119] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.043021] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.044042] Spectre V2 : User space: Vulnerable [ 0.045017] Speculative Store Bypass: Vulnerable [ 0.048296] debug: unmapping init [mem 0xffffffff9d859000-0xffffffff9d860fff] [ 0.050292] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.051770] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.052043] ... version: 2 [ 0.053023] ... bit width: 48 [ 0.054022] ... generic registers: 4 [ 0.055021] ... value mask: 0000ffffffffffff [ 0.056025] ... max period: 00007fffffffffff [ 0.057022] ... fixed-purpose events: 3 [ 0.058028] ... event mask: 000000070000000f [ 0.060345] rcu: Hierarchical SRCU implementation. [ 0.062878] smp: Bringing up secondary CPUs ... [ 0.063811] x86: Booting SMP configuration: [ 0.064043] .... node #0, CPUs: #1 #2 #3 [ 0.068113] smp: Brought up 1 node, 4 CPUs [ 0.070028] smpboot: Max logical packages: 1 [ 0.071029] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.302677] node 0 deferred pages initialised in 230ms [ 0.307131] devtmpfs: initialized [ 0.308338] x86/mm: Memory block size: 128MB [ 0.311493] gcov: version magic: 0x41383552 [ 0.313362] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.318161] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.321396] pinctrl core: initialized pinctrl subsystem [ 0.323824] ************************************************************* [ 0.326017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.329014] ** ** [ 0.332016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.335017] ** ** [ 0.338015] ** This means that this kernel is built to expose internal ** [ 0.341013] ** IOMMU data structures, which may compromise security on ** [ 0.344017] ** your system. ** [ 0.347019] ** ** [ 0.350014] ** If you see this message and you are not debugging the ** [ 0.353018] ** kernel, report this immediately to your vendor! ** [ 0.355014] ** ** [ 0.357013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.359015] ************************************************************* [ 0.361683] NET: Registered protocol family 16 [ 0.363530] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.366065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.368079] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.372088] cpuidle: using governor menu [ 0.374154] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.377705] PCI: Using configuration type 1 for base access [ 0.379149] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.389068] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.390055] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.392066] cryptd: max_cpu_qlen set to 1000 [ 0.394362] ACPI: Added _OSI(Module Device) [ 0.397030] ACPI: Added _OSI(Processor Device) [ 0.398019] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.400026] ACPI: Added _OSI(Processor Aggregator Device) [ 0.405360] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.414362] ACPI: Interpreter enabled [ 0.416092] ACPI: PM: (supports S0 S3 S4 S5) [ 0.418022] ACPI: Using IOAPIC for interrupt routing [ 0.420162] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.425537] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.436458] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.439045] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.441023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.444099] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.448559] acpiphp: Slot [2] registered [ 0.450161] acpiphp: Slot [5] registered [ 0.451148] acpiphp: Slot [6] registered [ 0.452111] acpiphp: Slot [7] registered [ 0.453125] acpiphp: Slot [8] registered [ 0.455116] acpiphp: Slot [9] registered [ 0.456139] acpiphp: Slot [10] registered [ 0.457113] acpiphp: Slot [3] registered [ 0.459118] acpiphp: Slot [4] registered [ 0.461556] acpiphp: Slot [11] registered [ 0.462139] acpiphp: Slot [12] registered [ 0.464166] acpiphp: Slot [13] registered [ 0.466171] acpiphp: Slot [14] registered [ 0.468237] acpiphp: Slot [15] registered [ 0.470145] acpiphp: Slot [16] registered [ 0.471169] acpiphp: Slot [17] registered [ 0.473159] acpiphp: Slot [18] registered [ 0.475166] acpiphp: Slot [19] registered [ 0.477229] acpiphp: Slot [20] registered [ 0.479202] acpiphp: Slot [21] registered [ 0.480190] acpiphp: Slot [22] registered [ 0.482174] acpiphp: Slot [23] registered [ 0.484142] acpiphp: Slot [24] registered [ 0.487202] acpiphp: Slot [25] registered [ 0.489162] acpiphp: Slot [26] registered [ 0.491182] acpiphp: Slot [27] registered [ 0.493136] acpiphp: Slot [28] registered [ 0.494121] acpiphp: Slot [29] registered [ 0.496150] acpiphp: Slot [30] registered [ 0.497104] acpiphp: Slot [31] registered [ 0.498061] PCI host bridge to bus 0000:00 [ 0.500022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.502029] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.504028] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.506030] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.509033] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.512029] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.513302] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.516048] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.520099] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.530018] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.535525] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.538022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.540024] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.542021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.546617] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.550556] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.555072] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.560165] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.566021] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.579036] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.586022] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.592231] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.599018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.606017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.624022] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.636842] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.645019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.654020] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.674024] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.689791] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.700023] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.707113] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.727027] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.738593] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.751017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.757021] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.774026] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.787335] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.794023] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.803021] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.819022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.829071] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.837016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.842014] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.856025] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.866353] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.869457] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.871471] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.874350] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.876183] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.882154] iommu: Default domain type: Passthrough [ 0.884519] SCSI subsystem initialized [ 0.886198] ACPI: bus type USB registered [ 0.888118] usbcore: registered new interface driver usbfs [ 0.890104] usbcore: registered new interface driver hub [ 0.892105] usbcore: registered new device driver usb [ 0.894194] pps_core: LinuxPPS API ver. 1 registered [ 0.896013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.900076] PTP clock support registered [ 0.902079] EDAC MC: Ver: 3.0.0 [ 0.904557] PCI: Using ACPI for IRQ routing [ 0.906025] PCI: pci_cache_line_size set to 64 bytes [ 0.906364] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] [ 0.906380] e820: reserve RAM buffer [mem 0xbffce000-0xbfffffff] [ 0.906384] e820: reserve RAM buffer [mem 0x146e00000-0x147ffffff] [ 0.906919] NetLabel: Initializing [ 0.909013] NetLabel: domain hash size = 128 [ 0.910013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.912069] NetLabel: unlabeled traffic allowed by default [ 0.914123] vgaarb: loaded [ 0.916343] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.918018] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.925006] clocksource: Switched to clocksource kvm-clock [ 1.040840] VFS: Disk quotas dquot_6.6.0 [ 1.042895] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.046284] *** VALIDATE ramfs *** [ 1.047765] *** VALIDATE hugetlbfs *** [ 1.050176] pnp: PnP ACPI init [ 1.051773] pnp 00:00: Plug and Play ACPI device, IDs PNP0303 (active) [ 1.051849] pnp 00:01: Plug and Play ACPI device, IDs PNP0f13 (active) [ 1.051877] pnp 00:02: [dma 2] [ 1.051946] pnp 00:02: Plug and Play ACPI device, IDs PNP0700 (active) [ 1.052034] pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) [ 1.052126] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) [ 1.052187] pnp 00:05: Plug and Play ACPI device, IDs PNP0b00 (active) [ 1.052877] pnp: PnP ACPI: found 6 devices [ 1.072766] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.076049] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.078387] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.081077] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.084033] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.086945] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.089444] NET: Registered protocol family 2 [ 1.091396] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.096749] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.101818] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.107855] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.111972] TCP: Hash tables configured (established 65536 bind 65536) [ 1.115643] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.119531] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.122751] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.126172] NET: Registered protocol family 1 [ 1.129229] RPC: Registered named UNIX socket transport module. [ 1.131313] RPC: Registered udp transport module. [ 1.132936] RPC: Registered tcp transport module. [ 1.135032] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.137185] NET: Registered protocol family 44 [ 1.139372] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.142372] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.145079] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.148049] PCI: CLS 0 bytes, default 64 [ 1.150174] Unpacking initramfs... [ 2.594235] debug: unmapping init [mem 0xffff892a3cc54000-0xffff892a3ffbffff] [ 2.599702] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.602482] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.605457] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.147817] Initialise system trusted keyrings [ 3.150070] Key type blacklist registered [ 3.152473] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.164747] zbud: loaded [ 3.168655] *** VALIDATE nfs *** [ 3.169630] *** VALIDATE nfs4 *** [ 3.171023] pstore: using deflate compression [ 3.174315] Platform Keyring initialized [ 3.297404] NET: Registered protocol family 38 [ 3.299799] Key type asymmetric registered [ 3.301648] Asymmetric key parser 'x509' registered [ 3.303890] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.307198] io scheduler mq-deadline registered [ 3.309424] io scheduler kyber registered [ 3.312435] io scheduler bfq registered [ 3.314864] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.318566] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.322333] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.326105] ACPI: Power Button [PWRF] [ 3.332765] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.338922] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.350610] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.357064] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.375026] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.404246] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.434237] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.440605] Non-volatile memory driver v1.3 [ 3.442718] Linux agpgart interface v0.103 [ 3.476673] virtio_blk virtio1: [vda] 67992 512-byte logical blocks (34.8 MB/33.2 MiB) [ 3.481469] vda: detected capacity change from 0 to 34811904 [ 3.498152] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.501816] vdb: detected capacity change from 0 to 1073741824 [ 3.518109] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.521728] vdc: detected capacity change from 0 to 2621440000 [ 3.538335] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.541631] vdd: detected capacity change from 0 to 2621440000 [ 3.559393] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.562441] vde: detected capacity change from 0 to 4294967296 [ 3.576905] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.580315] vdf: detected capacity change from 0 to 4294967296 [ 3.587527] libphy: Fixed MDIO Bus: probed [ 3.595094] usbcore: registered new interface driver usbserial_generic [ 3.598830] usbserial: USB Serial support registered for generic [ 3.601617] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.605576] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.607829] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.610582] mousedev: PS/2 mouse device common for all mice [ 3.615069] rtc_cmos 00:05: RTC can wake from S4 [ 3.617723] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.619314] rtc_cmos 00:05: registered as rtc0 [ 3.624267] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.625642] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.628558] intel_pstate: CPU model not supported [ 3.633357] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.638473] hid: raw HID events driver (C) Jiri Kosina [ 3.641294] usbcore: registered new interface driver usbhid [ 3.643969] usbhid: USB HID core driver [ 3.646107] drop_monitor: Initializing network drop monitor service [ 3.649447] Initializing XFRM netlink socket [ 3.652095] NET: Registered protocol family 10 [ 3.656085] Segment Routing with IPv6 [ 3.658149] NET: Registered protocol family 17 [ 3.660866] mpls_gso: MPLS GSO support [ 3.662585] start plist test [ 3.665898] end plist test [ 3.667737] RAS: Correctable Errors collector initialized. [ 3.670172] AVX version of gcm_enc/dec engaged. [ 3.672022] AES CTR mode by8 optimization enabled [ 3.747787] sched_clock: Marking stable (3747762215, 0)->(4705091139, -957328924) [ 3.751459] registered taskstats version 1 [ 3.753590] Loading compiled-in X.509 certificates [ 3.755871] zswap: loaded using pool lzo/zbud [ 3.779889] Key type big_key registered [ 3.791170] Key type encrypted registered [ 3.792622] ima: No TPM chip found, activating TPM-bypass! [ 3.794015] ima: Allocated hash algorithm: sha1 [ 3.795419] ima: No architecture policies found [ 3.796466] evm: Initialising EVM extended attributes: [ 3.797687] evm: security.selinux [ 3.798630] evm: security.ima [ 3.799533] evm: security.capability [ 3.800806] evm: HMAC attrs: 0x1 [ 3.803301] rtc_cmos 00:05: setting system clock to 2026-04-14 20:32:03 UTC (1776198723) [ 3.810805] debug: unmapping init [mem 0xffffffff9e803000-0xffffffff9e9fffff] [ 3.814779] debug: unmapping init [mem 0xffffffff9d582000-0xffffffff9d858fff] [ 3.823251] Write protecting the kernel read-only data: 28672k [ 3.827621] debug: unmapping init [mem 0xffffffff9bc03000-0xffffffff9bdfffff] [ 3.831160] debug: unmapping init [mem 0xffffffff9c514000-0xffffffff9c5fffff] [ 3.871471] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.881501] systemd[1]: Detected virtualization kvm. [ 3.883786] systemd[1]: Detected architecture x86-64. [ 3.886039] systemd[1]: Running in initial RAM disk. [ 3.914814] systemd[1]: No hostname configured. [ 3.916499] systemd[1]: Set hostname to . [ 3.918443] random: systemd: uninitialized urandom read (16 bytes read) [ 3.920629] systemd[1]: Initializing machine ID from random generator. [ 3.974136] random: ln: uninitialized urandom read (6 bytes read) [ 4.058901] random: systemd: uninitialized urandom read (16 bytes read) [ 4.061377] systemd[1]: Reached target Local File Systems. [ 4.076892] systemd[1]: Listening on udev Kernel Socket. [ 4.081811] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ 4.764493] device-mapper: uevent: version 1.0.3 [ 4.766959] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ 5.019051] random: fast init done [ 5.532348] virtio_net virtio0 ens2: renamed from eth0 [ 5.557418] libata version 3.00 loaded. [ 5.584832] ata_piix 0000:00:01.1: version 2.13 [ 5.643697] scsi host0: ata_piix [ 5.689137] scsi host1: ata_piix [ 5.691242] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.694074] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.306750] random: crng init done [ 10.308215] random: 7 urandom warning(s) missed due to ratelimiting [ 10.803467] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ 11.947180] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.202737] SELinux: Disabled at runtime. [ 12.265464] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.273398] systemd[1]: Detected virtualization kvm. [ 12.275707] systemd[1]: Detected architecture x86-64. [ 12.742473] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.745798] systemd[1]: Stopped Switch Root. [ 12.750466] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.752805] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.754848] systemd[1]: Stopped Journal Service. [ 12.763265] systemd[1]: Starting Journal Service... [ 12.781434] systemd[1]: Listening on Process Core Dump Socket. [ 12.832816] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ 13.186652] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 13.437785] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.471854] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.591920] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.603159] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 [ 13.603188] EDAC sbridge: Ver: 1.1.2 [ 15.258773] Key type dns_resolver registered [ 15.563489] NFS: Registering the id_resolver key type [ 15.565352] Key type id_resolver registered [ 15.567062] Key type id_legacy registered [ 37.862577] libcfs: loading out-of-tree module taints kernel. [ 37.884047] alg: No test for adler32 (adler32-zlib) [ 38.635382] Key type ._llcrypt registered [ 38.637135] Key type .llcrypt registered [ 38.691464] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_hostid [ 55.060630] hrtimer: interrupt took 3619682 ns [ 57.930406] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing load_modules_local [ 59.835849] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 60.712689] Lustre: Lustre: Build Version: 2.15.8_2_g69e1f89 [ 61.793814] LNet: Added LNI 192.168.204.117@tcp [8/256/0/180] [ 61.797600] LNet: Accept secure, port 988 [ 63.674079] Key type lgssc registered [ 65.699555] Lustre: Echo OBD driver; http://www.lustre.org/ [ 86.178732] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 90.117671] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 102.713367] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 111.170072] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 120.157484] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 138.066542] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing load_modules_local [ 153.055869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 153.164401] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 154.483995] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 154.521868] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 154.649339] Lustre: lustre-MDT0000: new disk, initializing [ 154.767130] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 154.794975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 159.615944] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 174.942729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 175.019115] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 175.103990] Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 [ 175.154834] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space: rc = -61 [ 175.161550] Lustre: Skipped 1 previous similar message [ 175.292708] Lustre: lustre-MDT0001: new disk, initializing [ 175.391966] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 175.435459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 175.449789] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 180.609540] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 194.242645] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 194.316285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 194.323435] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [ 194.508685] Lustre: lustre-OST0000: new disk, initializing [ 194.513028] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 194.574408] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 199.914483] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 203.327696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 203.340278] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 214.211407] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 214.296563] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 214.305931] Lustre: lustre-OST0001-osd: enabled 'large_dir' feature on device /dev/mapper/ost2_flakey [ 214.447835] Lustre: lustre-OST0001: new disk, initializing [ 214.453680] Lustre: srv-lustre-OST0001: No data found on store. Initialize space: rc = -61 [ 214.519254] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 218.793426] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 221.714200] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 221.727838] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 231.061404] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 242.207810] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 251.989312] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing check_logdir /tmp/testlogs/ [ 258.534708] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing yml_node [ 263.264746] Lustre: DEBUG MARKER: Client: 2.15.8.2 [ 266.320440] Lustre: DEBUG MARKER: MDS: 2.15.8.2 [ 269.381205] Lustre: DEBUG MARKER: OSS: 2.15.8.2 [ 271.453509] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity ============----- Tue Apr 14 16:36:29 EDT 2026 [ 281.706219] Lustre: DEBUG MARKER: - need MDS1_VERSION < v2_14_55-100-g8a84c7f9c7 (34539522 < 34486116) for LU-14927, skip 0f [ 283.570211] Lustre: DEBUG MARKER: excepting tests: 225 255 256 400a 42a 42b 42c 407 118c 118d 817 411 [ 285.315810] Lustre: DEBUG MARKER: skipping tests SLOW=no: 27m 60i 64b 68 71 115 135 136 230d 300o [ 291.841986] Lustre: DEBUG MARKER: oleg417-client.virtnet: executing check_config_client /mnt/lustre [ 313.212327] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 316.700111] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 320.570202] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 333.061968] Lustre: DEBUG MARKER: == sanity test 200: OST pools ============================ 16:37:30 (1776199050) [ 372.866923] Lustre: DEBUG MARKER: == sanity test 204a: Print default stripe attributes ===== 16:38:10 (1776199090) [ 383.075597] Lustre: DEBUG MARKER: == sanity test 204b: Print default stripe size and offset ========================================================== 16:38:20 (1776199100) [ 392.070318] Lustre: DEBUG MARKER: == sanity test 204c: Print default stripe count and offset ========================================================== 16:38:30 (1776199110) [ 399.771271] Lustre: DEBUG MARKER: == sanity test 204d: Print default stripe count and size ========================================================== 16:38:37 (1776199117) [ 408.232237] Lustre: DEBUG MARKER: == sanity test 204e: Print raw stripe attributes ========= 16:38:45 (1776199125) [ 416.388072] Lustre: DEBUG MARKER: == sanity test 204f: Print raw stripe size and offset ==== 16:38:54 (1776199134) [ 423.941822] Lustre: DEBUG MARKER: == sanity test 204g: Print raw stripe count and offset === 16:39:02 (1776199142) [ 431.930942] Lustre: DEBUG MARKER: == sanity test 204h: Print raw stripe count and size ===== 16:39:10 (1776199150) [ 441.370222] Lustre: DEBUG MARKER: == sanity test 205a: Verify job stats ==================== 16:39:19 (1776199159) [ 448.315900] Lustre: lustre-MDD0000: changelog on [ 451.432362] Lustre: lustre-MDD0001: changelog on [ 455.832935] Lustre: DEBUG MARKER: Test: /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 -c 1 /mnt/lustre/d205a.sanity [ 458.373475] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.lfs.11424 [ 461.945121] Lustre: DEBUG MARKER: Test: rmdir /mnt/lustre/d205a.sanity [ 463.616199] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.rmdir.24890 [ 466.946844] Lustre: DEBUG MARKER: Test: lfs mkdir -i 1 /mnt/lustre/d205a.sanity.remote [ 468.669330] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.lfs.23267 [ 472.845849] Lustre: DEBUG MARKER: Test: mknod /mnt/lustre/f205a.sanity c 1 3 [ 475.002673] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.mknod.28866 [ 479.010934] Lustre: DEBUG MARKER: Test: rm -f /mnt/lustre/f205a.sanity [ 481.093646] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.rm.2692 [ 484.548904] Lustre: DEBUG MARKER: Test: /home/green/git/lustre-release/lustre/utils/lfs setstripe -i 0 -c 1 /mnt/lustre/f205a.sanity [ 487.791932] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.lfs.7599 [ 491.679431] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity [ 493.443281] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.touch.2971 [ 497.355549] Lustre: DEBUG MARKER: Test: dd if=/dev/zero of=/mnt/lustre/f205a.sanity bs=1M count=1 oflag=sync [ 499.519332] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.dd.24304 [ 501.951279] Lustre: DEBUG MARKER: Test: dd if=/mnt/lustre/f205a.sanity of=/dev/null bs=1M count=1 iflag=direct [ 503.772874] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.dd.8445 [ 506.266849] Lustre: DEBUG MARKER: Test: /home/green/git/lustre-release/lustre/tests/truncate /mnt/lustre/f205a.sanity 0 [ 508.474869] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.truncate.24953 [ 512.275568] Lustre: DEBUG MARKER: Test: mv -f /mnt/lustre/f205a.sanity /mnt/lustre/d205a.sanity.rename [ 514.428862] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.mv.3422 [ 517.405564] Lustre: DEBUG MARKER: Test: /home/green/git/lustre-release/lustre/utils/lfs mkdir -i 0 -c 1 /mnt/lustre/d205a.sanity.expire [ 519.225697] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.lfs.4509 [ 527.523047] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity [ 529.301602] Lustre: DEBUG MARKER: Using JobID environment USER=S.root.touch.0.oleg417-client.v [ 532.360601] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity [ 533.999836] Lustre: DEBUG MARKER: Using JobID environment USER=S.root.touch.0.oleg417-client.E [ 536.745516] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity [ 538.199538] Lustre: DEBUG MARKER: Using JobID environment session=S.root.touch.0.oleg417-client.v [ 547.373930] Lustre: lustre-MDD0001: changelog off [ 550.205824] Lustre: lustre-MDD0000: changelog off [ 554.096718] Lustre: DEBUG MARKER: == sanity test 205b: Verify job stats jobid and output format ========================================================== 16:41:11 (1776199271) [ 567.778482] Lustre: DEBUG MARKER: == sanity test 205c: Verify client stats format ========== 16:41:25 (1776199285) [ 574.672131] Lustre: DEBUG MARKER: == sanity test 206: fail lov_init_raid0() doesn't lbug === 16:41:32 (1776199292) [ 581.756495] Lustre: DEBUG MARKER: == sanity test 207a: can refresh layout at glimpse ======= 16:41:40 (1776199300) [ 590.037439] Lustre: DEBUG MARKER: == sanity test 207b: can refresh layout at open ========== 16:41:47 (1776199307) [ 597.807451] Lustre: DEBUG MARKER: == sanity test 208: Exclusive open ======================= 16:41:56 (1776199316) [ 606.634053] Lustre: Failing over lustre-MDT0000 [ 606.876069] Lustre: server umount lustre-MDT0000 complete [ 608.471358] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 192.168.204.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 610.811751] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 610.822711] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 610.823105] LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 610.828210] Lustre: Skipped 2 previous similar messages [ 610.896538] LustreError: Skipped 3 previous similar messages [ 613.558417] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 192.168.204.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 615.904416] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 615.935728] LustreError: Skipped 3 previous similar messages [ 621.025269] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 621.047626] LustreError: Skipped 4 previous similar messages [ 628.580729] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 628.697926] LustreError: 11-0: MGC192.168.204.117@tcp: operation mgs_target_reg to node 0@lo failed: rc = -107 [ 628.702607] LustreError: 166-1: MGC192.168.204.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 628.717736] LustreError: 3376:0:(import.c:702:ptlrpc_connect_import_locked()) already connecting [ 628.722994] Lustre: Evicted from MGS (at 192.168.204.117@tcp) after server handle changed from 0xd2f59beca4558f25 to 0xd2f59beca455e0fc [ 628.746463] Lustre: MGC192.168.204.117@tcp: Connection restored to (at 0@lo) [ 628.918498] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.17@tcp (not set up) [ 629.042127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 629.111367] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 630.578254] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 634.221805] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 634.343250] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to (at 0@lo) [ 634.403688] Lustre: lustre-MDT0000: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 634.466212] Lustre: lustre-OST0001: deleting orphan objects from 0x0:8 to 0x0:33 [ 634.477183] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17 to 0x0:33 [ 644.380119] Lustre: DEBUG MARKER: oleg417-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 646.197276] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 652.572928] Lustre: Failing over lustre-MDT0000 [ 652.755183] Lustre: server umount lustre-MDT0000 complete [ 654.816042] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 654.827537] Lustre: Skipped 3 previous similar messages [ 654.830244] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 654.855499] LustreError: Skipped 12 previous similar messages [ 661.471218] Lustre: 3375:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1776199374/real 1776199374] req@000000006927f597 x1862479413149184/t0(0) o400->MGC192.168.204.117@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1776199381 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u8:2.0' [ 661.506584] LustreError: 166-1: MGC192.168.204.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 672.745553] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 672.768298] LustreError: Skipped 24 previous similar messages [ 674.343770] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 678.887887] Lustre: Evicted from MGS (at 192.168.204.117@tcp) after server handle changed from 0xd2f59beca455e0fc to 0xd2f59beca455e4fa [ 678.908424] Lustre: MGC192.168.204.117@tcp: Connection restored to (at 0@lo) [ 678.916188] Lustre: Skipped 3 previous similar messages [ 679.180736] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 679.224288] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 680.108316] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 682.861193] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 684.526262] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to (at 0@lo) [ 684.571279] Lustre: lustre-MDT0000: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 684.673240] Lustre: lustre-OST0001: deleting orphan objects from 0x0:8 to 0x0:65 [ 684.673240] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17 to 0x0:65 [ 695.515119] Lustre: DEBUG MARKER: oleg417-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 697.988649] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 710.436927] Lustre: DEBUG MARKER: == sanity test 209: read-only open/close requests should be freed promptly ========================================================== 16:43:48 (1776199428) [ 735.850173] Lustre: DEBUG MARKER: == sanity test 210: lfs getstripe does not break leases == 16:44:13 (1776199453) [ 747.526923] Lustre: DEBUG MARKER: == sanity test 212: Sendfile test ====================================================================================================== 16:44:25 (1776199465) [ 758.693274] Lustre: DEBUG MARKER: == sanity test 213: OSC lock completion and cancel race don't crash - bug 18829 ========================================================== 16:44:36 (1776199476) [ 778.724809] Lustre: DEBUG MARKER: == sanity test 214: hash-indexed directory test - bug 20133 ========================================================== 16:44:56 (1776199496) [ 815.694578] Lustre: DEBUG MARKER: == sanity test 215: lnet exists and has proper content - bugs 18102, 21079, 21517 ========================================================== 16:45:33 (1776199533) [ 824.961693] Lustre: DEBUG MARKER: == sanity test 216: check lockless direct write updates file size and kms correctly ========================================================== 16:45:42 (1776199542) [ 856.236679] Lustre: DEBUG MARKER: == sanity test 217: check lctl ping for hostnames with hiphen ('-') ========================================================== 16:46:13 (1776199573) [ 866.615580] Lustre: DEBUG MARKER: == sanity test 218: parallel read and truncate should not deadlock ========================================================== 16:46:24 (1776199584) [ 869.210737] Lustre: DEBUG MARKER: creating a 10 Mb file [ 872.078776] Lustre: DEBUG MARKER: starting reads [ 874.446687] Lustre: DEBUG MARKER: truncating the file [ 877.430426] Lustre: DEBUG MARKER: killing dd [ 879.465115] Lustre: DEBUG MARKER: removing the temporary file [ 887.348853] Lustre: DEBUG MARKER: == sanity test 219: LU-394: Write partial won't cause uncontiguous pages vec at LND ========================================================== 16:46:45 (1776199605) [ 896.700723] Lustre: DEBUG MARKER: == sanity test 220: preallocated MDS objects still used if ENOSPC from OST ========================================================== 16:46:54 (1776199614) [ 902.623854] Lustre: *** cfs_fail_loc=229, val=1*** [ 905.184115] Lustre: *** cfs_fail_loc=229, val=1*** [ 907.748663] Lustre: *** cfs_fail_loc=229, val=1*** [ 907.752850] Lustre: Skipped 2 previous similar messages [ 909.869496] Lustre: *** cfs_fail_loc=229, val=0*** [ 928.462470] Lustre: DEBUG MARKER: == sanity test 221: make sure fault and truncate race to not cause OOM ========================================================== 16:47:26 (1776199646) [ 943.388476] Lustre: DEBUG MARKER: == sanity test 222a: AGL for ls should not trigger CLIO lock failure ========================================================== 16:47:41 (1776199661) [ 953.556774] Lustre: DEBUG MARKER: == sanity test 222b: AGL for rmdir should not trigger CLIO lock failure ========================================================== 16:47:51 (1776199671) [ 963.021832] Lustre: DEBUG MARKER: == sanity test 223: osc reenqueue if without AGL lock granted ================================================================================= 16:48:00 (1776199680) [ 971.844797] Lustre: DEBUG MARKER: == sanity test 224a: Don't panic on bulk IO failure ====== 16:48:09 (1776199689) [ 972.102821] Lustre: lustre-OST0000: Client 9031a013-aff2-4506-b7c4-2e0eeb5144b9 (at 192.168.204.17@tcp) reconnecting [ 985.483758] Lustre: DEBUG MARKER: == sanity test 224b: Don't panic on bulk IO failure ====== 16:48:23 (1776199703) [ 991.052588] LustreError: 25134:0:(fail.c:138:__cfs_fail_timeout_set()) cfs_fail_timeout id 515 sleeping for 3000ms [ 994.082856] LustreError: 25134:0:(fail.c:149:__cfs_fail_timeout_set()) cfs_fail_timeout id 515 awake [ 1004.540825] Lustre: DEBUG MARKER: == sanity test 224c: Don't hang if one of md lost during large bulk RPC ========================================================== 16:48:42 (1776199722) [ 1017.616376] Lustre: *** cfs_fail_loc=520, val=57344*** [ 1017.722338] LNet: *** cfs_fail_loc=e000, val=2147483648*** [ 1022.695936] Lustre: lustre-OST0000: Client 9031a013-aff2-4506-b7c4-2e0eeb5144b9 (at 192.168.204.17@tcp) reconnecting [ 1039.967232] Lustre: DEBUG MARKER: == sanity test 224d: Don't corrupt data on bulk IO timeout ========================================================== 16:49:17 (1776199757) [ 1046.111838] LustreError: 25134:0:(fail.c:138:__cfs_fail_timeout_set()) cfs_fail_timeout id 515 sleeping for 22000ms [ 1066.217290] Lustre: lustre-OST0000: Client 9031a013-aff2-4506-b7c4-2e0eeb5144b9 (at 192.168.204.17@tcp) reconnecting [ 1068.135111] LustreError: 25134:0:(fail.c:149:__cfs_fail_timeout_set()) cfs_fail_timeout id 515 awake [ 1068.140154] LustreError: 25134:0:(ldlm_lib.c:3493:target_bulk_io()) @@@ bulk READ failed: rc = -107 req@0000000012778b27 x1862479392469376/t0(0) o3->9031a013-aff2-4506-b7c4-2e0eeb5144b9@192.168.204.17@tcp:0/0 lens 488/440 e 0 to 0 dl 1776199785 ref 1 fl Interpret:/0/0 rc 0/0 job:'dd.0' [ 1068.164259] Lustre: 25134:0:(service.c:2348:ptlrpc_server_handle_request()) @@@ Request took longer than estimated (20/2s); client may timeout req@0000000012778b27 x1862479392469376/t0(0) o3->9031a013-aff2-4506-b7c4-2e0eeb5144b9@192.168.204.17@tcp:0/0 lens 488/440 e 0 to 0 dl 1776199785 ref 1 fl Complete:/0/0 rc 0/0 job:'dd.0' [ 1078.850439] Lustre: DEBUG MARKER: SKIP: sanity test_225a skipping excluded test 225a (base 225) [ 1080.779328] Lustre: DEBUG MARKER: SKIP: sanity test_225b skipping excluded test 225b (base 225) [ 1082.790545] Lustre: DEBUG MARKER: == sanity test 226a: call path2fid and fid2path on files of all type ========================================================== 16:50:00 (1776199800) [ 1092.027406] Lustre: DEBUG MARKER: == sanity test 226b: call path2fid and fid2path on files of all type under remote dir ========================================================== 16:50:09 (1776199809) [ 1100.632526] Lustre: DEBUG MARKER: == sanity test 226c: call path2fid and fid2path under remote dir with subdir mount ========================================================== 16:50:18 (1776199818) [ 1101.386482] Lustre: lustre-MDT0000: subdir mount '/d226c.sanity' is remote and may be slow [ 1109.204568] Lustre: DEBUG MARKER: == sanity test 227: running truncated executable does not cause OOM ========================================================== 16:50:27 (1776199827) [ 1118.364761] Lustre: DEBUG MARKER: == sanity test 228a: try to reuse idle OI blocks ========= 16:50:36 (1776199836) [ 1446.583358] Lustre: DEBUG MARKER: == sanity test 228b: idle OI blocks can be reused after MDT restart ========================================================== 16:56:04 (1776200164) [ 1722.863871] Lustre: Failing over lustre-MDT0000 [ 1723.391101] Lustre: server umount lustre-MDT0000 complete [ 1723.872912] LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1723.883192] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1723.902130] Lustre: Skipped 1 previous similar message [ 1723.916806] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1723.925850] LustreError: Skipped 10 previous similar messages [ 1724.901513] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1724.914681] Lustre: Skipped 2 previous similar messages [ 1730.020403] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1730.044320] LustreError: Skipped 8 previous similar messages [ 1732.066888] Lustre: 3373:0:(client.c:2295:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1776200444/real 1776200444] req@000000008f6857b9 x1862479414792512/t0(0) o400->MGC192.168.204.117@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1776200451 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u8:1.0' [ 1732.114286] LustreError: 166-1: MGC192.168.204.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1732.609982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1738.215620] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1738.218982] Lustre: Evicted from MGS (at 192.168.204.117@tcp) after server handle changed from 0xd2f59beca455e4fa to 0xd2f59beca46ce95d [ 1738.231337] LustreError: Skipped 13 previous similar messages [ 1738.251481] Lustre: MGC192.168.204.117@tcp: Connection restored to (at 0@lo) [ 1738.259126] Lustre: Skipped 3 previous similar messages [ 1738.495491] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1738.547085] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1739.818577] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 1743.864026] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to (at 0@lo) [ 1743.913346] Lustre: lustre-MDT0000: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 1743.928740] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 1743.966500] Lustre: lustre-OST0000: deleting orphan objects from 0x0:6275 to 0x0:6305 [ 1743.970886] Lustre: lustre-OST0001: deleting orphan objects from 0x0:6243 to 0x0:6273 [ 1803.961622] Lustre: DEBUG MARKER: == sanity test 228c: NOT shrink the last entry in OI index node to recycle idle leaf ========================================================== 17:02:02 (1776200522) [ 2387.563181] Lustre: DEBUG MARKER: == sanity test 229: getstripe/stat/rm/attr changes work on released files ========================================================== 17:11:46 (1776201106) [ 2394.838473] Lustre: DEBUG MARKER: == sanity test 230a: Create remote directory and files under the remote directory ========================================================== 17:11:52 (1776201112) [ 2403.832974] Lustre: DEBUG MARKER: == sanity test 230b: migrate directory =================== 17:12:01 (1776201121) [ 2467.580766] Lustre: DEBUG MARKER: == sanity test 230c: check directory accessiblity if migration failed ========================================================== 17:13:05 (1776201185) [ 2469.643897] Lustre: *** cfs_fail_loc=1801, val=0*** [ 2471.880055] LustreError: 9828:0:(mdd_dir.c:4234:mdd_migrate_cmd_check()) lustre-MDD0000: 'migrate_dir' migration was interrupted, run 'lfs migrate -m 1 -c 1 -H crush migrate_dir' to finish migration. [ 2480.069930] Lustre: DEBUG MARKER: SKIP: sanity test_230d skipping SLOW test 230d [ 2482.065664] Lustre: DEBUG MARKER: == sanity test 230e: migrate mulitple local link files === 17:13:20 (1776201200) [ 2491.108158] Lustre: DEBUG MARKER: == sanity test 230f: migrate mulitple remote link files == 17:13:29 (1776201209) [ 2501.951651] Lustre: DEBUG MARKER: == sanity test 230g: migrate dir to non-exist MDT ======== 17:13:39 (1776201219) [ 2510.492245] Lustre: DEBUG MARKER: == sanity test 230h: migrate .. and root ================= 17:13:48 (1776201228) [ 2518.953086] Lustre: DEBUG MARKER: == sanity test 230i: lfs migrate -m tolerates trailing slashes ========================================================== 17:13:56 (1776201236) [ 2528.848349] Lustre: DEBUG MARKER: == sanity test 230j: DoM file data not changed after dir migration ========================================================== 17:14:06 (1776201246) [ 2537.915551] Lustre: DEBUG MARKER: == sanity test 230k: file data not changed after dir migration ========================================================== 17:14:15 (1776201255) [ 2539.620117] Lustre: DEBUG MARKER: SKIP: sanity test_230k needs >= 4 MDTs [ 2542.113956] Lustre: DEBUG MARKER: == sanity test 230l: readdir between MDTs won't crash ==== 17:14:19 (1776201259) [ 2689.036917] Lustre: DEBUG MARKER: == sanity test 230m: xattrs not changed after dir migration ========================================================== 17:16:47 (1776201407) [ 2705.467794] Lustre: DEBUG MARKER: == sanity test 230n: Dir migration with mirrored file ==== 17:17:03 (1776201423) [ 2714.118549] Lustre: DEBUG MARKER: == sanity test 230o: dir split =========================== 17:17:12 (1776201432) [ 2741.005603] Lustre: DEBUG MARKER: == sanity test 230p: dir merge =========================== 17:17:39 (1776201459) [ 2767.019874] Lustre: DEBUG MARKER: == sanity test 230q: dir auto split ====================== 17:18:05 (1776201485) [ 2816.462779] Lustre: DEBUG MARKER: == sanity test 230r: migrate with too many local locks === 17:18:54 (1776201534) [ 2825.282537] Lustre: DEBUG MARKER: == sanity test 230s: lfs mkdir should return -EEXIST if target exists ========================================================== 17:19:03 (1776201543) [ 2837.750963] Lustre: DEBUG MARKER: == sanity test 230t: migrate directory with project ID set ========================================================== 17:19:15 (1776201555) [ 2846.044632] Lustre: DEBUG MARKER: == sanity test 230u: migrate directory by QOS ============ 17:19:24 (1776201564) [ 2847.833427] Lustre: DEBUG MARKER: SKIP: sanity test_230u needs >= 4 MDTs [ 2849.643410] Lustre: DEBUG MARKER: == sanity test 230v: subdir migrated to the MDT where its parent is located ========================================================== 17:19:27 (1776201567) [ 2851.664752] Lustre: DEBUG MARKER: SKIP: sanity test_230v needs >= 4 MDTs [ 2853.863836] Lustre: DEBUG MARKER: == sanity test 230w: non-recursive mode dir migration ==== 17:19:31 (1776201571) [ 2862.543983] Lustre: DEBUG MARKER: == sanity test 230y: unlink dir with bad hash type ======= 17:19:40 (1776201580) [ 2866.103367] Lustre: *** cfs_fail_loc=1802, val=0*** [ 2866.624447] Lustre: *** cfs_fail_loc=1802, val=0*** [ 2866.629526] Lustre: Skipped 5 previous similar messages [ 2867.685276] Lustre: *** cfs_fail_loc=1802, val=0*** [ 2867.686977] Lustre: Skipped 13 previous similar messages [ 2869.741779] Lustre: *** cfs_fail_loc=1802, val=0*** [ 2869.743811] Lustre: Skipped 22 previous similar messages [ 2873.777933] Lustre: *** cfs_fail_loc=1802, val=0*** [ 2873.780907] Lustre: Skipped 41 previous similar messages [ 2890.657235] Lustre: DEBUG MARKER: == sanity test 230z: resume dir migration with bad hash type ========================================================== 17:20:08 (1776201608) [ 2894.225693] Lustre: *** cfs_fail_loc=1802, val=0*** [ 2894.228310] Lustre: Skipped 8 previous similar messages [ 2953.259679] Lustre: DEBUG MARKER: == sanity test 231a: checking that reading/writing of BRW RPC size results in one RPC ========================================================== 17:21:11 (1776201671) [ 2965.907759] Lustre: DEBUG MARKER: == sanity test 231b: must not assert on fully utilized OST request buffer ========================================================== 17:21:23 (1776201683) [ 3005.095443] Lustre: DEBUG MARKER: == sanity test 232a: failed lock should not block umount ========================================================== 17:22:03 (1776201723) [ 3006.170141] Lustre: *** cfs_fail_loc=31c, val=0*** [ 3013.351775] Lustre: Failing over lustre-OST0000 [ 3013.523024] Lustre: server umount lustre-OST0000 complete [ 3014.112407] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3014.122041] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3014.166013] LustreError: 137-5: lustre-OST0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3014.627496] LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3014.647273] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3016.672946] LustreError: 137-5: lustre-OST0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3016.710355] LustreError: Skipped 3 previous similar messages [ 3020.974831] LustreError: 137-5: lustre-OST0000_UUID: not available for connect from 192.168.204.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3025.025108] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3025.448360] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 3025.468996] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 3026.112350] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3027.369649] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 3027.369705] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 3027.382812] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17865 to 0x0:17889 [ 3027.418768] Lustre: Skipped 3 previous similar messages [ 3027.439826] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:6034 to 0x280000400:6049 [ 3031.741561] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 3041.592871] Lustre: DEBUG MARKER: == sanity test 232b: failed data version lock should not block umount ========================================================== 17:22:39 (1776201759) [ 3043.009938] Lustre: *** cfs_fail_loc=31c, val=0*** [ 3043.021722] LustreError: 30719:0:(ldlm_request.c:487:ldlm_cli_enqueue_local()) ### delayed lvb init failed (rc -12) ns: filter-lustre-OST0000_UUID lock: 00000000bf1d7537/0xd2f59beca48b2a90 lrc: 2/0,0 mode: --/PR res: [0x45e2:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 30719 timeout: 0 lvb_type: 0 [ 3049.025639] Lustre: Failing over lustre-OST0000 [ 3049.053075] Lustre: server umount lustre-OST0000 complete [ 3050.980741] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3051.000679] LustreError: 137-5: lustre-OST0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3051.023813] LustreError: Skipped 3 previous similar messages [ 3057.855576] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3058.009281] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 3058.020296] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 3059.482505] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3059.940682] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 3059.945767] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 3059.987174] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:6034 to 0x280000400:6081 [ 3059.996627] Lustre: Skipped 1 previous similar message [ 3060.030968] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17891 to 0x0:17921 [ 3063.273380] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 3075.014822] Lustre: DEBUG MARKER: == sanity test 233a: checking that OBF of the FS root succeeds ========================================================== 17:23:12 (1776201792) [ 3085.170693] Lustre: DEBUG MARKER: == sanity test 233b: checking that OBF of the FS .lustre succeeds ========================================================== 17:23:22 (1776201802) [ 3095.943344] Lustre: DEBUG MARKER: == sanity test 234: xattr cache should not crash on ENOMEM ========================================================== 17:23:33 (1776201813) [ 3105.466490] Lustre: DEBUG MARKER: == sanity test 235: LU-1715: flock deadlock detection does not work properly ========================================================== 17:23:43 (1776201823) [ 3115.420979] Lustre: DEBUG MARKER: == sanity test 236: Layout swap on open unlinked file ==== 17:23:53 (1776201833) [ 3124.157925] Lustre: DEBUG MARKER: == sanity test 238: Verify linkea consistency ============ 17:24:02 (1776201842) [ 3132.806765] Lustre: DEBUG MARKER: == sanity test 239A: osp_sync test ======================= 17:24:10 (1776201850) [ 3262.543152] Lustre: DEBUG MARKER: == sanity test 239a: process invalid osp sync record correctly ========================================================== 17:26:21 (1776201981) [ 3265.606343] Lustre: *** cfs_fail_loc=2100, val=0*** [ 3275.248197] Lustre: DEBUG MARKER: == sanity test 239b: process osp sync record with ENOMEM error correctly ========================================================== 17:26:33 (1776201993) [ 3291.198118] Lustre: DEBUG MARKER: == sanity test 240: race between ldlm enqueue and the connection RPC (no ASSERT) ========================================================== 17:26:49 (1776202009) [ 3295.023762] Lustre: *** cfs_fail_loc=713, val=0*** [ 3295.034724] Lustre: Skipped 1 previous similar message [ 3305.173920] Lustre: DEBUG MARKER: == sanity test 241a: bio vs dio ========================== 17:27:03 (1776202023) [ 3402.037115] Lustre: DEBUG MARKER: == sanity test 241b: dio vs dio ========================== 17:28:40 (1776202120) [ 3445.918968] Lustre: DEBUG MARKER: == sanity test 242: mdt_readpage failure should not cause directory unreadable ========================================================== 17:29:23 (1776202163) [ 3447.201081] Lustre: *** cfs_fail_loc=105, val=0*** [ 3456.539794] Lustre: DEBUG MARKER: == sanity test 243: various group lock tests ============= 17:29:33 (1776202173) [ 3553.383090] Lustre: DEBUG MARKER: == sanity test 244a: sendfile with group lock tests ====== 17:31:11 (1776202271) [ 3576.059745] Lustre: DEBUG MARKER: == sanity test 244b: multi-threaded write with group lock ========================================================== 17:31:34 (1776202294) [ 3582.289120] Lustre: DEBUG MARKER: == sanity test 245: check mdc connection flag/data: multiple modify RPCs ========================================================== 17:31:40 (1776202300) [ 3587.978155] Lustre: DEBUG MARKER: == sanity test 247a: mount subdir as fileset ============= 17:31:46 (1776202306) [ 3598.465471] Lustre: DEBUG MARKER: == sanity test 247b: mount subdir that dose not exist ==== 17:31:56 (1776202316) [ 3605.697601] Lustre: DEBUG MARKER: == sanity test 247c: running fid2path outside subdirectory root ========================================================== 17:32:04 (1776202324) [ 3607.223108] Lustre: lustre-MDT0000: subdir mount '/d247c.sanity' is remote and may be slow [ 3618.221113] Lustre: DEBUG MARKER: == sanity test 247d: running fid2path inside subdirectory root ========================================================== 17:32:16 (1776202336) [ 3619.392809] Lustre: lustre-MDT0000: subdir mount '/d247d.sanity' is remote and may be slow [ 3628.561527] Lustre: DEBUG MARKER: == sanity test 247e: mount .. as fileset ================= 17:32:26 (1776202346) [ 3636.719513] Lustre: DEBUG MARKER: == sanity test 247f: mount striped or remote directory as fileset ========================================================== 17:32:34 (1776202354) [ 3641.491308] Lustre: lustre-MDT0000: subdir mount '/d247f.sanity/remote' refused because 'enable_remote_subdir_mount=0': rc = -66 [ 3655.398660] Lustre: lustre-MDT0000: subdir mount '/d247f.sanity/remote' is remote and may be slow [ 3665.323750] Lustre: DEBUG MARKER: == sanity test 247g: mount striped directory as fileset caches ROOT lookup lock ========================================================== 17:33:03 (1776202383) [ 3666.962222] Lustre: DEBUG MARKER: SKIP: sanity test_247g needs >= 4 MDTs [ 3668.813706] Lustre: DEBUG MARKER: == sanity test 248a: fast read verification ============== 17:33:07 (1776202387) [ 3699.299478] Lustre: DEBUG MARKER: == sanity test 248b: test short_io read and write for both small and large sizes ========================================================== 17:33:37 (1776202417) [ 3747.551987] Lustre: DEBUG MARKER: == sanity test 249: Write above 2T file size ============= 17:34:25 (1776202465) [ 3753.872732] Lustre: DEBUG MARKER: == sanity test 250: Write above 16T limit ================ 17:34:32 (1776202472) [ 3760.705898] Lustre: DEBUG MARKER: == sanity test 251: Handling short read and write correctly ========================================================== 17:34:38 (1776202478) [ 3767.276531] Lustre: DEBUG MARKER: == sanity test 252: check lr_reader tool ================= 17:34:45 (1776202485) [ 3779.270265] Lustre: DEBUG MARKER: == sanity test 253: Check object allocation limit ======== 17:34:57 (1776202497) [ 3851.759096] LustreError: 6279:0:(lod_qos.c:1362:lod_ost_alloc_specific()) can't lstripe objid [0x200001b74:0x22:0x0]: have 0 want 1 [ 3919.463076] Lustre: DEBUG MARKER: == sanity test 254: Check changelog size ================= 17:37:17 (1776202637) [ 3923.239711] Lustre: lustre-MDD0000: changelog on [ 3926.124870] Lustre: lustre-MDD0001: changelog on [ 3936.623610] Lustre: lustre-MDD0001: changelog off [ 3939.037370] Lustre: lustre-MDD0000: changelog off [ 3941.615866] Lustre: DEBUG MARKER: SKIP: sanity test_255a skipping excluded test 255a (base 255) [ 3943.111346] Lustre: DEBUG MARKER: SKIP: sanity test_255b skipping excluded test 255b (base 255) [ 3944.876747] Lustre: DEBUG MARKER: SKIP: sanity test_255c skipping excluded test 255c (base 255) [ 3946.553487] Lustre: DEBUG MARKER: SKIP: sanity test_256 skipping excluded test 256 [ 3948.210273] Lustre: DEBUG MARKER: == sanity test 257: xattr locks are not lost ============= 17:37:46 (1776202666) [ 3949.985531] Lustre: *** cfs_fail_loc=161, val=0*** [ 3951.551938] Lustre: Failing over lustre-MDT0001 [ 3952.337170] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.17@tcp (stopping) [ 3954.146548] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3954.156979] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3954.158591] Lustre: Skipped 2 previous similar messages [ 3954.174131] Lustre: Skipped 2 previous similar messages [ 3956.990047] Lustre: server umount lustre-MDT0001 complete [ 3957.419832] LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 192.168.204.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3957.433830] LustreError: Skipped 4 previous similar messages [ 3962.545854] LustreError: 137-5: lustre-MDT0001_UUID: not available for connect from 192.168.204.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3962.560761] LustreError: Skipped 3 previous similar messages [ 3965.037094] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3965.431495] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 3965.473871] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 3967.465161] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 3969.912439] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 3970.548948] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to (at 0@lo) [ 3970.555215] Lustre: Skipped 1 previous similar message [ 3970.582651] Lustre: lustre-MDT0001: Recovery over after 0:03, of 2 clients 2 recovered and 0 were evicted. [ 3970.618437] Lustre: lustre-OST0001: deleting orphan objects from 0x2c0000400:8517 to 0x2c0000400:8545 [ 3970.618516] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8588 to 0x280000400:8609 [ 3974.834325] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing _wait_recovery_complete *.lustre-MDT0001.recovery_status 1475 [ 3980.100407] Lustre: DEBUG MARKER: == sanity test 258a: verify i_mutex security behavior when suid attributes is set ========================================================== 17:38:18 (1776202698) [ 3986.153783] Lustre: DEBUG MARKER: == sanity test 258b: verify i_mutex security behavior ==== 17:38:24 (1776202704) [ 3992.664531] Lustre: DEBUG MARKER: == sanity test 259: crash at delayed truncate ============ 17:38:30 (1776202710) [ 4010.658048] Lustre: *** cfs_fail_loc=2301, val=0*** [ 4013.152560] Lustre: Failing over lustre-OST0000 [ 4013.186775] Lustre: server umount lustre-OST0000 complete [ 4013.780586] LustreError: 137-5: lustre-OST0000_UUID: not available for connect from 192.168.204.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4013.790699] LustreError: Skipped 3 previous similar messages [ 4015.072755] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4015.087213] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4015.104969] Lustre: Skipped 1 previous similar message [ 4016.612964] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4022.562577] LDISKFS-fs (dm-2): 1 truncate cleaned up [ 4022.565272] LDISKFS-fs (dm-2): recovery complete [ 4022.576439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4022.750605] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4022.774239] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4023.940054] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4024.029775] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4024.034167] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 4024.055165] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8588 to 0x280000400:8641 [ 4024.065979] Lustre: Skipped 2 previous similar messages [ 4024.103580] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17943 to 0x0:17985 [ 4026.668099] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4038.373518] Lustre: DEBUG MARKER: == sanity test 260: Check mdc_close fail ================= 17:39:16 (1776202756) [ 4045.288612] Lustre: DEBUG MARKER: == sanity test 270a: DoM: basic functionality tests ====== 17:39:23 (1776202763) [ 4060.430218] Lustre: DEBUG MARKER: == sanity test 270b: DoM: maximum size overflow checks for DoM-only file ========================================================== 17:39:38 (1776202778) [ 4067.983079] Lustre: DEBUG MARKER: == sanity test 270c: DoM: DoM EA inheritance tests ======= 17:39:46 (1776202786) [ 4075.136953] Lustre: DEBUG MARKER: == sanity test 270d: DoM: change striping from DoM to RAID0 ========================================================== 17:39:53 (1776202793) [ 4082.596492] Lustre: DEBUG MARKER: == sanity test 270e: DoM: lfs find with DoM files test === 17:40:00 (1776202800) [ 4092.503211] Lustre: DEBUG MARKER: == sanity test 270f: DoM: maximum DoM stripe size checks ========================================================== 17:40:10 (1776202810) [ 4098.792410] Lustre: Increasing provided stripe size to a minimum value 64 [ 4110.372830] Lustre: DEBUG MARKER: == sanity test 270g: DoM: default DoM stripe size depends on free space ========================================================== 17:40:28 (1776202828) [ 4117.970569] Lustre: *** cfs_fail_loc=168, val=0*** [ 4119.800823] Lustre: *** cfs_fail_loc=168, val=0*** [ 4119.805852] Lustre: Skipped 2 previous similar messages [ 4121.471378] Lustre: *** cfs_fail_loc=168, val=0*** [ 4121.473271] Lustre: Skipped 2 previous similar messages [ 4125.013409] Lustre: *** cfs_fail_loc=168, val=0*** [ 4125.016775] Lustre: Skipped 5 previous similar messages [ 4136.710398] Lustre: DEBUG MARKER: == sanity test 270h: DoM: DoM stripe removal when disabled on server ========================================================== 17:40:54 (1776202854) [ 4146.890468] Lustre: DEBUG MARKER: == sanity test 270i: DoM: setting invalid DoM striping should fail ========================================================== 17:41:05 (1776202865) [ 4153.803588] Lustre: DEBUG MARKER: == sanity test 271a: DoM: data is cached for read after write ========================================================== 17:41:11 (1776202871) [ 4160.658419] Lustre: DEBUG MARKER: == sanity test 271b: DoM: no glimpse RPC for stat (DoM only file) ========================================================== 17:41:18 (1776202878) [ 4167.074776] Lustre: DEBUG MARKER: == sanity test 271ba: DoM: no glimpse RPC for stat (combined file) ========================================================== 17:41:25 (1776202885) [ 4173.834494] Lustre: DEBUG MARKER: == sanity test 271c: DoM: IO lock at open saves enqueue RPCs ========================================================== 17:41:32 (1776202892) [ 4296.775187] Lustre: DEBUG MARKER: == sanity test 271d: DoM: read on open (1K file in reply buffer) ========================================================== 17:43:34 (1776203014) [ 4304.955904] Lustre: DEBUG MARKER: == sanity test 271f: DoM: read on open (200K file and read tail) ========================================================== 17:43:43 (1776203023) [ 4313.542546] Lustre: DEBUG MARKER: == sanity test 271g: Discard DoM data vs client flush race ========================================================== 17:43:51 (1776203031) [ 4322.190034] Lustre: DEBUG MARKER: == sanity test 272a: DoM migration: new layout with the same DOM component ========================================================== 17:44:00 (1776203040) [ 4330.732307] Lustre: DEBUG MARKER: == sanity test 272b: DoM migration: DOM file to the OST-striped file (plain) ========================================================== 17:44:08 (1776203048) [ 4341.991407] Lustre: DEBUG MARKER: == sanity test 272c: DoM migration: DOM file to the OST-striped file (composite) ========================================================== 17:44:20 (1776203060) [ 4352.237779] Lustre: DEBUG MARKER: == sanity test 272d: DoM mirroring: OST-striped mirror to DOM file ========================================================== 17:44:30 (1776203070) [ 4362.408808] Lustre: DEBUG MARKER: == sanity test 272e: DoM mirroring: DOM mirror to the OST-striped file ========================================================== 17:44:40 (1776203080) [ 4370.910601] Lustre: DEBUG MARKER: == sanity test 272f: DoM migration: OST-striped file to DOM file ========================================================== 17:44:49 (1776203089) [ 4378.973464] Lustre: DEBUG MARKER: == sanity test 273a: DoM: layout swapping should fail with DOM ========================================================== 17:44:57 (1776203097) [ 4387.767855] Lustre: DEBUG MARKER: == sanity test 273b: DoM: race writeback and object destroy ========================================================== 17:45:05 (1776203105) [ 4389.726511] LustreError: 63729:0:(fail.c:138:__cfs_fail_timeout_set()) cfs_fail_timeout id 16b sleeping for 2000ms [ 4391.727155] LustreError: 63729:0:(fail.c:149:__cfs_fail_timeout_set()) cfs_fail_timeout id 16b awake [ 4400.547312] Lustre: DEBUG MARKER: == sanity test 275: Read on a canceled duplicate lock ==== 17:45:18 (1776203118) [ 4411.234240] Lustre: DEBUG MARKER: == sanity test 276: Race between mount and obd_statfs ==== 17:45:29 (1776203129) [ 4413.897593] Lustre: Failing over lustre-OST0000 [ 4413.900900] LustreError: 67027:0:(obd_class.h:1061:obd_statfs()) Device 18 not setup [ 4413.996585] Lustre: server umount lustre-OST0000 complete [ 4416.233229] LustreError: 137-5: lustre-OST0000_UUID: not available for connect from 192.168.204.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4416.265677] LustreError: Skipped 6 previous similar messages [ 4416.993844] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4424.552393] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4424.886797] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4424.912970] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4426.060286] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4426.345717] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4426.345766] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 4426.378173] Lustre: Skipped 1 previous similar message [ 4426.384185] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18017 [ 4426.422063] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:8705 [ 4430.723558] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4435.097318] Lustre: Failing over lustre-OST0000 [ 4435.101986] LustreError: 69275:0:(obd_class.h:1061:obd_statfs()) Device 18 not setup [ 4435.177316] Lustre: server umount lustre-OST0000 complete [ 4435.432670] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4435.454469] Lustre: Skipped 1 previous similar message [ 4435.465708] LustreError: 137-5: lustre-OST0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4435.500904] LustreError: Skipped 6 previous similar messages [ 4445.163871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4445.285450] LustreError: 70334:0:(obd_class.h:1061:obd_statfs()) Device 18 not setup [ 4445.377523] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4445.391485] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4446.555234] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4446.928117] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 4446.928311] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4446.939782] Lustre: Skipped 1 previous similar message [ 4446.958738] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18049 [ 4446.964112] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:8737 [ 4451.247508] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4455.894486] Lustre: Failing over lustre-OST0000 [ 4455.904507] LustreError: 71449:0:(obd_class.h:1061:obd_statfs()) Device 18 not setup [ 4455.915899] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4455.931987] Lustre: Skipped 1 previous similar message [ 4455.934727] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4455.993475] Lustre: server umount lustre-OST0000 complete [ 4465.479285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4465.834265] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4465.868316] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4467.371364] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4467.866153] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4467.877173] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 4467.898051] Lustre: Skipped 1 previous similar message [ 4467.932668] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18081 [ 4467.950687] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:8769 [ 4471.583399] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4476.063297] Lustre: Failing over lustre-OST0000 [ 4476.067957] LustreError: 73667:0:(obd_class.h:1061:obd_statfs()) Device 18 not setup [ 4476.112206] Lustre: server umount lustre-OST0000 complete [ 4476.386177] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4476.403127] Lustre: Skipped 1 previous similar message [ 4476.416567] LustreError: 137-5: lustre-OST0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4476.436203] LustreError: Skipped 9 previous similar messages [ 4485.158894] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4485.362116] LustreError: 74761:0:(obd_class.h:1061:obd_statfs()) Device 18 not setup [ 4485.473516] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4485.492845] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4486.587546] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4487.021797] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4487.030042] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 4487.034377] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18113 [ 4487.037556] Lustre: Skipped 1 previous similar message [ 4487.052336] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:8801 [ 4490.232283] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4493.826931] Lustre: Failing over lustre-OST0000 [ 4493.864423] Lustre: server umount lustre-OST0000 complete [ 4502.554620] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4502.838183] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4502.855534] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4503.222733] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4504.073377] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4504.082267] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 4504.107105] Lustre: Skipped 1 previous similar message [ 4504.118090] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18145 [ 4504.127438] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:8833 [ 4509.136874] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4513.772994] LustreError: 78086:0:(obd_class.h:1061:obd_statfs()) Device 18 not setup [ 4513.779605] LustreError: 78086:0:(obd_class.h:1061:obd_statfs()) Skipped 1 previous similar message [ 4514.273276] LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4514.293850] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4514.329180] Lustre: Skipped 3 previous similar messages [ 4523.864732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4525.437835] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:8865 [ 4525.450234] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18177 [ 4529.839839] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4534.143632] Lustre: Failing over lustre-OST0000 [ 4534.149126] Lustre: Skipped 1 previous similar message [ 4534.250457] Lustre: server umount lustre-OST0000 complete [ 4534.257883] Lustre: Skipped 1 previous similar message [ 4542.791860] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4543.041661] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4543.046584] Lustre: Skipped 1 previous similar message [ 4543.055955] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4543.065136] Lustre: Skipped 1 previous similar message [ 4544.206272] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4544.222155] Lustre: Skipped 1 previous similar message [ 4544.790521] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4544.793342] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 4544.798283] Lustre: Skipped 1 previous similar message [ 4544.805953] Lustre: Skipped 3 previous similar messages [ 4544.819679] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18209 [ 4544.828244] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:8897 [ 4547.747906] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4551.273769] LustreError: 82486:0:(obd_class.h:1061:obd_statfs()) Device 18 not setup [ 4551.278224] LustreError: 82486:0:(obd_class.h:1061:obd_statfs()) Skipped 3 previous similar messages [ 4553.191947] LustreError: 137-5: lustre-OST0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4553.207882] LustreError: Skipped 23 previous similar messages [ 4562.320871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4564.498486] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:8929 [ 4564.512938] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18241 [ 4568.348841] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4574.687851] LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4574.695479] LustreError: Skipped 2 previous similar messages [ 4584.102423] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4586.047525] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:8961 [ 4586.054622] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18273 [ 4589.931954] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4594.667115] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4594.678658] Lustre: Skipped 7 previous similar messages [ 4603.145984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4604.999735] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:8993 [ 4605.020781] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18305 [ 4607.972781] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4612.915840] Lustre: Failing over lustre-OST0000 [ 4612.925213] Lustre: Skipped 3 previous similar messages [ 4612.984377] Lustre: server umount lustre-OST0000 complete [ 4612.993349] Lustre: Skipped 3 previous similar messages [ 4623.367978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4623.840728] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4623.850210] Lustre: Skipped 3 previous similar messages [ 4623.879514] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4623.901312] Lustre: Skipped 3 previous similar messages [ 4625.308510] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4625.326940] Lustre: Skipped 3 previous similar messages [ 4625.666417] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4625.668125] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 4625.685175] Lustre: Skipped 3 previous similar messages [ 4625.724735] Lustre: Skipped 7 previous similar messages [ 4625.733453] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18337 [ 4625.747595] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9025 [ 4631.292088] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4635.616875] LustreError: 91289:0:(obd_class.h:1061:obd_statfs()) Device 18 not setup [ 4635.622484] LustreError: 91289:0:(obd_class.h:1061:obd_statfs()) Skipped 5 previous similar messages [ 4636.130420] LustreError: 11-0: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4645.451035] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4647.063355] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18369 [ 4647.066477] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9057 [ 4650.652857] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4665.897881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4667.437105] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9089 [ 4667.439040] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18401 [ 4673.195562] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4677.629168] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4677.644489] LustreError: Skipped 2 previous similar messages [ 4681.701641] LustreError: 137-5: lustre-OST0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4681.725177] LustreError: Skipped 41 previous similar messages [ 4687.809295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4689.864499] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9121 [ 4689.884440] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18433 [ 4695.069599] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4700.128470] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4700.138501] LustreError: Skipped 1 previous similar message [ 4708.695974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4710.695234] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9153 [ 4710.704954] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18465 [ 4714.424405] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4729.837248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4731.993185] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18497 [ 4731.996778] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9185 [ 4738.162596] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4744.928806] Lustre: Failing over lustre-OST0000 [ 4744.936778] Lustre: Skipped 5 previous similar messages [ 4745.049425] Lustre: server umount lustre-OST0000 complete [ 4745.054054] Lustre: Skipped 5 previous similar messages [ 4745.698062] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4745.736083] Lustre: Skipped 13 previous similar messages [ 4757.078059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4757.392451] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4757.413411] Lustre: Skipped 5 previous similar messages [ 4757.426627] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4757.443639] Lustre: Skipped 5 previous similar messages [ 4758.514700] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4758.528640] Lustre: Skipped 5 previous similar messages [ 4758.607398] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4758.611298] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.117@tcp (at 0@lo) [ 4758.613578] Lustre: Skipped 5 previous similar messages [ 4758.623607] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18529 [ 4758.624363] Lustre: Skipped 11 previous similar messages [ 4758.640522] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9217 [ 4762.918734] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4776.895808] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4777.063311] LustreError: 105663:0:(obd_class.h:1061:obd_statfs()) Device 18 not setup [ 4777.075502] LustreError: 105663:0:(obd_class.h:1061:obd_statfs()) Skipped 8 previous similar messages [ 4778.943202] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9249 [ 4778.952156] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18561 [ 4782.641614] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4799.034445] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4801.127300] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18593 [ 4801.127300] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9281 [ 4805.669764] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4818.570666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4820.659323] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9313 [ 4820.666140] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18625 [ 4824.487422] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4834.979758] Lustre: DEBUG MARKER: == sanity test 277: Direct IO shall drop page cache ====== 17:52:33 (1776203553) [ 4842.779648] Lustre: DEBUG MARKER: == sanity test 278: Race starting MDS between MDTs stop/start ========================================================== 17:52:40 (1776203560) [ 4846.404229] LustreError: 11-0: lustre-MDT0000-lwp-MDT0001: operation mds_disconnect to node 0@lo failed: rc = -107 [ 4846.685467] LustreError: 111687:0:(libcfs_fail.h:169:cfs_race()) cfs_race id 60c sleeping [ 4852.191748] LustreError: 111687:0:(libcfs_fail.h:178:cfs_race()) cfs_fail_race id 60c awake: rc=0 [ 4856.462965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4856.567917] LustreError: 166-1: MGC192.168.204.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4856.607315] Lustre: Evicted from MGS (at 192.168.204.117@tcp) after server handle changed from 0xd2f59beca46ce95d to 0xd2f59beca4961f75 [ 4856.698481] LustreError: 112250:0:(libcfs_fail.h:180:cfs_race()) cfs_fail_race id 60c waking [ 4859.097334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4864.148420] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4874.138437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4874.641671] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4879.885425] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4880.761805] Lustre: lustre-OST0001: deleting orphan objects from 0x0:17861 to 0x0:17889 [ 4880.762951] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18657 [ 4880.846235] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:8661 to 0x280000400:9345 [ 4886.467476] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing _wait_recovery_complete *.lustre-MDT0001.recovery_status 1475 [ 4894.448280] Lustre: DEBUG MARKER: == sanity test 280: Race between MGS umount and client llog processing ========================================================== 17:53:32 (1776203612) [ 4898.316243] LustreError: 112257:0:(libcfs_fail.h:169:cfs_race()) cfs_race id 15e sleeping [ 4901.305282] LustreError: 114110:0:(libcfs_fail.h:180:cfs_race()) cfs_fail_race id 15e waking [ 4901.315876] LustreError: 112257:0:(libcfs_fail.h:178:cfs_race()) cfs_fail_race id 15e awake: rc=2009 [ 4901.347174] LustreError: 11-0: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4901.359194] LustreError: Skipped 2 previous similar messages [ 4902.379855] LustreError: 114110:0:(libcfs_fail.h:180:cfs_race()) cfs_fail_race id 15e waking [ 4911.339924] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4911.470062] LustreError: 166-1: MGC192.168.204.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4911.481415] LustreError: 114559:0:(import.c:702:ptlrpc_connect_import_locked()) already connecting [ 4911.483467] Lustre: Evicted from MGS (at 192.168.204.117@tcp) after server handle changed from 0xd2f59beca4961f75 to 0xd2f59beca4962525 [ 4911.515177] LustreError: 114567:0:(libcfs_fail.h:180:cfs_race()) cfs_fail_race id 15e waking [ 4912.751946] LustreError: 114564:0:(libcfs_fail.h:180:cfs_race()) cfs_fail_race id 15e waking [ 4913.826628] LustreError: 114564:0:(libcfs_fail.h:180:cfs_race()) cfs_fail_race id 15e waking [ 4915.888869] LustreError: 114566:0:(libcfs_fail.h:180:cfs_race()) cfs_fail_race id 15e waking [ 4915.895529] LustreError: 114566:0:(libcfs_fail.h:180:cfs_race()) Skipped 1 previous similar message [ 4920.034424] LustreError: 114567:0:(libcfs_fail.h:180:cfs_race()) cfs_fail_race id 15e waking [ 4920.049799] LustreError: 114567:0:(libcfs_fail.h:180:cfs_race()) Skipped 3 previous similar messages [ 4922.171782] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4924.450237] Lustre: lustre-OST0000: deleting orphan objects from 0x0:17998 to 0x0:18689 [ 4924.450518] Lustre: lustre-OST0001: deleting orphan objects from 0x0:17861 to 0x0:17921 [ 4927.548866] Lustre: DEBUG MARKER: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 [ 4929.922434] LustreError: 114567:0:(libcfs_fail.h:180:cfs_race()) cfs_fail_race id 15e waking [ 4929.939434] LustreError: 114567:0:(libcfs_fail.h:180:cfs_race()) Skipped 1 previous similar message [ 4944.191967] Lustre: DEBUG MARKER: == sanity test 300a: basic striped dir sanity test ======= 17:54:21 (1776203661) [ 4954.038286] Lustre: DEBUG MARKER: == sanity test 300b: check ctime/mtime for striped dir === 17:54:31 (1776203671) [ 4985.741944] Lustre: DEBUG MARKER: == sanity test 300c: chown [ 5126.125657] Lustre: DEBUG MARKER: == sanity test 300d: check default stripe under striped directory ========================================================== 17:57:24 (1776203844) [ 5292.706430] Lustre: DEBUG MARKER: sanity test_300d: @@@@@@ FAIL: wrong stripe 1 for /mnt/lustre/d300d.sanity/striped_dir/f3