Aug 15 05:51:55 oleg133-client kernel: Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 Aug 15 05:51:55 oleg133-client kernel: Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 Aug 15 05:51:55 oleg133-client kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' Aug 15 05:51:55 oleg133-client kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' Aug 15 05:51:55 oleg133-client kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' Aug 15 05:51:55 oleg133-client kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 Aug 15 05:51:55 oleg133-client kernel: x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. Aug 15 05:51:55 oleg133-client kernel: signal: max sigframe size: 1776 Aug 15 05:51:55 oleg133-client kernel: BIOS-provided physical RAM map: Aug 15 05:51:55 oleg133-client kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable Aug 15 05:51:55 oleg133-client kernel: BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved Aug 15 05:51:55 oleg133-client kernel: BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved Aug 15 05:51:55 oleg133-client kernel: BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable Aug 15 05:51:55 oleg133-client kernel: BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved Aug 15 05:51:55 oleg133-client kernel: BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved Aug 15 05:51:55 oleg133-client kernel: BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved Aug 15 05:51:55 oleg133-client kernel: BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable Aug 15 05:51:55 oleg133-client kernel: NX (Execute Disable) protection: active Aug 15 05:51:55 oleg133-client kernel: SMBIOS 2.8 present. Aug 15 05:51:55 oleg133-client kernel: DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 05:51:55 oleg133-client kernel: Hypervisor detected: KVM Aug 15 05:51:55 oleg133-client kernel: kvm-clock: Using msrs 4b564d01 and 4b564d00 Aug 15 05:51:55 oleg133-client kernel: kvm-clock: using sched offset of 505192773 cycles Aug 15 05:51:55 oleg133-client kernel: clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns Aug 15 05:51:55 oleg133-client kernel: tsc: Detected 2399.998 MHz processor Aug 15 05:51:55 oleg133-client kernel: e820: update [mem 0x00000000-0x00000fff] usable ==> reserved Aug 15 05:51:55 oleg133-client kernel: e820: remove [mem 0x000a0000-0x000fffff] usable Aug 15 05:51:55 oleg133-client kernel: last_pfn = 0x146e00 max_arch_pfn = 0x400000000 Aug 15 05:51:55 oleg133-client kernel: MTRR default type: write-back Aug 15 05:51:55 oleg133-client kernel: MTRR fixed ranges enabled: Aug 15 05:51:55 oleg133-client kernel: 00000-9FFFF write-back Aug 15 05:51:55 oleg133-client kernel: A0000-BFFFF uncachable Aug 15 05:51:55 oleg133-client kernel: C0000-FFFFF write-protect Aug 15 05:51:55 oleg133-client kernel: MTRR variable ranges enabled: Aug 15 05:51:55 oleg133-client kernel: 0 base 0000C0000000 mask 3FFFC0000000 uncachable Aug 15 05:51:55 oleg133-client kernel: 1 disabled Aug 15 05:51:55 oleg133-client kernel: 2 disabled Aug 15 05:51:55 oleg133-client kernel: 3 disabled Aug 15 05:51:55 oleg133-client kernel: 4 disabled Aug 15 05:51:55 oleg133-client kernel: 5 disabled Aug 15 05:51:55 oleg133-client kernel: 6 disabled Aug 15 05:51:55 oleg133-client kernel: 7 disabled Aug 15 05:51:55 oleg133-client kernel: x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT Aug 15 05:51:55 oleg133-client kernel: last_pfn = 0xbffda max_arch_pfn = 0x400000000 Aug 15 05:51:55 oleg133-client kernel: found SMP MP-table at [mem 0x000f54d0-0x000f54df] Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a01000, 0x24a01fff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a02000, 0x24a02fff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a03000, 0x24a03fff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a04000, 0x24a04fff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a05000, 0x24a05fff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a06000, 0x24a06fff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a07000, 0x24a07fff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a08000, 0x24a08fff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a09000, 0x24a09fff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a0a000, 0x24a0afff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a0b000, 0x24a0bfff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: BRK [0x24a0c000, 0x24a0cfff] PGTABLE Aug 15 05:51:55 oleg133-client kernel: RAMDISK: [mem 0xbcc64000-0xbffcffff] Aug 15 05:51:55 oleg133-client kernel: ACPI: Early table checksum verification disabled Aug 15 05:51:55 oleg133-client kernel: ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) Aug 15 05:51:55 oleg133-client kernel: ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 15 05:51:55 oleg133-client kernel: ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 15 05:51:55 oleg133-client kernel: ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 15 05:51:55 oleg133-client kernel: ACPI: FACS 0x00000000BFFE0000 000040 Aug 15 05:51:55 oleg133-client kernel: ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) Aug 15 05:51:55 oleg133-client kernel: ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 15 05:51:55 oleg133-client kernel: ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 15 05:51:55 oleg133-client kernel: ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] Aug 15 05:51:55 oleg133-client kernel: ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] Aug 15 05:51:55 oleg133-client kernel: ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] Aug 15 05:51:55 oleg133-client kernel: ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] Aug 15 05:51:55 oleg133-client kernel: ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] Aug 15 05:51:55 oleg133-client kernel: ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] Aug 15 05:51:55 oleg133-client kernel: ACPI: Local APIC address 0xfee00000 Aug 15 05:51:55 oleg133-client kernel: No NUMA configuration found Aug 15 05:51:55 oleg133-client kernel: Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] Aug 15 05:51:55 oleg133-client kernel: NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] Aug 15 05:51:55 oleg133-client kernel: Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) Aug 15 05:51:55 oleg133-client kernel: Zone ranges: Aug 15 05:51:55 oleg133-client kernel: DMA [mem 0x0000000000001000-0x0000000000ffffff] Aug 15 05:51:55 oleg133-client kernel: DMA32 [mem 0x0000000001000000-0x00000000ffffffff] Aug 15 05:51:55 oleg133-client kernel: Normal [mem 0x0000000100000000-0x0000000146dfffff] Aug 15 05:51:55 oleg133-client kernel: Device empty Aug 15 05:51:55 oleg133-client kernel: Movable zone start for each node Aug 15 05:51:55 oleg133-client kernel: Early memory node ranges Aug 15 05:51:55 oleg133-client kernel: node 0: [mem 0x0000000000001000-0x000000000009efff] Aug 15 05:51:55 oleg133-client kernel: node 0: [mem 0x0000000000100000-0x00000000bffd9fff] Aug 15 05:51:55 oleg133-client kernel: node 0: [mem 0x0000000100000000-0x0000000146dfffff] Aug 15 05:51:55 oleg133-client kernel: Zeroed struct page in unavailable ranges: 4744 pages Aug 15 05:51:55 oleg133-client kernel: Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] Aug 15 05:51:55 oleg133-client kernel: On node 0 totalpages: 1076600 Aug 15 05:51:55 oleg133-client kernel: DMA zone: 64 pages used for memmap Aug 15 05:51:55 oleg133-client kernel: DMA zone: 158 pages reserved Aug 15 05:51:55 oleg133-client kernel: DMA zone: 3998 pages, LIFO batch:0 Aug 15 05:51:55 oleg133-client kernel: DMA32 zone: 12224 pages used for memmap Aug 15 05:51:55 oleg133-client kernel: DMA32 zone: 782298 pages, LIFO batch:63 Aug 15 05:51:55 oleg133-client kernel: Normal zone: 4536 pages used for memmap Aug 15 05:51:55 oleg133-client kernel: Normal zone: 290304 pages, LIFO batch:63 Aug 15 05:51:55 oleg133-client kernel: ACPI: PM-Timer IO Port: 0x608 Aug 15 05:51:55 oleg133-client kernel: ACPI: Local APIC address 0xfee00000 Aug 15 05:51:55 oleg133-client kernel: ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) Aug 15 05:51:55 oleg133-client kernel: IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 Aug 15 05:51:55 oleg133-client kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) Aug 15 05:51:55 oleg133-client kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) Aug 15 05:51:55 oleg133-client kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) Aug 15 05:51:55 oleg133-client kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) Aug 15 05:51:55 oleg133-client kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) Aug 15 05:51:55 oleg133-client kernel: ACPI: IRQ0 used by override. Aug 15 05:51:55 oleg133-client kernel: ACPI: IRQ5 used by override. Aug 15 05:51:55 oleg133-client kernel: ACPI: IRQ9 used by override. Aug 15 05:51:55 oleg133-client kernel: ACPI: IRQ10 used by override. Aug 15 05:51:55 oleg133-client kernel: ACPI: IRQ11 used by override. Aug 15 05:51:55 oleg133-client kernel: Using ACPI (MADT) for SMP configuration information Aug 15 05:51:55 oleg133-client kernel: ACPI: HPET id: 0x8086a201 base: 0xfed00000 Aug 15 05:51:55 oleg133-client kernel: TSC deadline timer available Aug 15 05:51:55 oleg133-client kernel: smpboot: Allowing 4 CPUs, 0 hotplug CPUs Aug 15 05:51:55 oleg133-client kernel: kvm-guest: KVM setup pv remote TLB flush Aug 15 05:51:55 oleg133-client kernel: kvm-guest: setup PV sched yield Aug 15 05:51:55 oleg133-client kernel: PM: Registered nosave memory: [mem 0x00000000-0x00000fff] Aug 15 05:51:55 oleg133-client kernel: PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] Aug 15 05:51:55 oleg133-client kernel: PM: Registered nosave memory: [mem 0x000a0000-0x000effff] Aug 15 05:51:55 oleg133-client kernel: PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] Aug 15 05:51:55 oleg133-client kernel: PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] Aug 15 05:51:55 oleg133-client kernel: PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] Aug 15 05:51:55 oleg133-client kernel: PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] Aug 15 05:51:55 oleg133-client kernel: PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] Aug 15 05:51:55 oleg133-client kernel: PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] Aug 15 05:51:55 oleg133-client kernel: [mem 0xc0000000-0xfeffbfff] available for PCI devices Aug 15 05:51:55 oleg133-client kernel: Booting paravirtualized kernel on KVM Aug 15 05:51:55 oleg133-client kernel: clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns Aug 15 05:51:55 oleg133-client kernel: setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 Aug 15 05:51:55 oleg133-client kernel: percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 Aug 15 05:51:55 oleg133-client kernel: pcpu-alloc: s221184 r8192 d28672 u524288 alloc=1*2097152 Aug 15 05:51:55 oleg133-client kernel: pcpu-alloc: [0] 0 1 2 3 Aug 15 05:51:55 oleg133-client kernel: kvm-guest: PV spinlocks enabled Aug 15 05:51:55 oleg133-client kernel: PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) Aug 15 05:51:55 oleg133-client kernel: Built 1 zonelists, mobility grouping on. Total pages: 1059618 Aug 15 05:51:55 oleg133-client kernel: Policy zone: Normal Aug 15 05:51:55 oleg133-client kernel: Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 Aug 15 05:51:55 oleg133-client kernel: Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. Aug 15 05:51:55 oleg133-client kernel: audit: disabled (until reboot) Aug 15 05:51:55 oleg133-client kernel: software IO TLB: area num 4. Aug 15 05:51:55 oleg133-client kernel: Memory: 2829700K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) Aug 15 05:51:55 oleg133-client kernel: SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 Aug 15 05:51:55 oleg133-client kernel: kmemleak: Kernel memory leak detector disabled Aug 15 05:51:55 oleg133-client kernel: ftrace: allocating 41240 entries in 162 pages Aug 15 05:51:55 oleg133-client kernel: ftrace: allocated 162 pages with 3 groups Aug 15 05:51:55 oleg133-client kernel: rcu: Hierarchical RCU implementation. Aug 15 05:51:55 oleg133-client kernel: rcu: #011RCU event tracing is enabled. Aug 15 05:51:55 oleg133-client kernel: rcu: #011RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. Aug 15 05:51:55 oleg133-client kernel: rcu: #011RCU callback double-/use-after-free debug enabled. Aug 15 05:51:55 oleg133-client kernel: #011Rude variant of Tasks RCU enabled. Aug 15 05:51:55 oleg133-client kernel: #011Tracing variant of Tasks RCU enabled. Aug 15 05:51:55 oleg133-client kernel: rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. Aug 15 05:51:55 oleg133-client kernel: rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 Aug 15 05:51:55 oleg133-client kernel: NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 Aug 15 05:51:55 oleg133-client kernel: random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 Aug 15 05:51:55 oleg133-client kernel: Console: colour *CGA 80x25 Aug 15 05:51:55 oleg133-client kernel: printk: console [ttyS1] enabled Aug 15 05:51:55 oleg133-client kernel: ACPI: Core revision 20220331 Aug 15 05:51:55 oleg133-client kernel: clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns Aug 15 05:51:55 oleg133-client kernel: hpet clockevent registered Aug 15 05:51:55 oleg133-client kernel: APIC: Switch to symmetric I/O mode setup Aug 15 05:51:55 oleg133-client kernel: x2apic enabled Aug 15 05:51:55 oleg133-client kernel: Switched APIC routing to physical x2apic. Aug 15 05:51:55 oleg133-client kernel: kvm-guest: setup PV IPIs Aug 15 05:51:55 oleg133-client kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 Aug 15 05:51:55 oleg133-client kernel: clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns Aug 15 05:51:55 oleg133-client kernel: Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) Aug 15 05:51:55 oleg133-client kernel: pid_max: default: 32768 minimum: 301 Aug 15 05:51:55 oleg133-client kernel: LSM: Security Framework initializing Aug 15 05:51:55 oleg133-client kernel: Yama: becoming mindful. Aug 15 05:51:55 oleg133-client kernel: SELinux: Initializing. Aug 15 05:51:55 oleg133-client kernel: *** VALIDATE selinux *** Aug 15 05:51:55 oleg133-client kernel: Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: *** VALIDATE tmpfs *** Aug 15 05:51:55 oleg133-client kernel: *** VALIDATE proc *** Aug 15 05:51:55 oleg133-client kernel: *** VALIDATE cgroup *** Aug 15 05:51:55 oleg133-client kernel: *** VALIDATE cgroup2 *** Aug 15 05:51:55 oleg133-client kernel: x86/cpu: User Mode Instruction Prevention (UMIP) activated Aug 15 05:51:55 oleg133-client kernel: Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 Aug 15 05:51:55 oleg133-client kernel: Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 Aug 15 05:51:55 oleg133-client kernel: Spectre V2 : User space: Vulnerable Aug 15 05:51:55 oleg133-client kernel: Speculative Store Bypass: Vulnerable Aug 15 05:51:55 oleg133-client kernel: debug: unmapping init [mem 0xffffffff9c259000-0xffffffff9c260fff] Aug 15 05:51:55 oleg133-client kernel: smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) Aug 15 05:51:55 oleg133-client kernel: Performance Events: IvyBridge events, full-width counters, Intel PMU driver. Aug 15 05:51:55 oleg133-client kernel: ... version: 2 Aug 15 05:51:55 oleg133-client kernel: ... bit width: 48 Aug 15 05:51:55 oleg133-client kernel: ... generic registers: 4 Aug 15 05:51:55 oleg133-client kernel: ... value mask: 0000ffffffffffff Aug 15 05:51:55 oleg133-client kernel: ... max period: 00007fffffffffff Aug 15 05:51:55 oleg133-client kernel: ... fixed-purpose events: 3 Aug 15 05:51:55 oleg133-client kernel: ... event mask: 000000070000000f Aug 15 05:51:55 oleg133-client kernel: rcu: Hierarchical SRCU implementation. Aug 15 05:51:55 oleg133-client kernel: smp: Bringing up secondary CPUs ... Aug 15 05:51:55 oleg133-client kernel: x86: Booting SMP configuration: Aug 15 05:51:55 oleg133-client kernel: .... node #0, CPUs: #1 #2 #3 Aug 15 05:51:55 oleg133-client kernel: smp: Brought up 1 node, 4 CPUs Aug 15 05:51:55 oleg133-client kernel: smpboot: Max logical packages: 1 Aug 15 05:51:55 oleg133-client kernel: smpboot: Total of 4 processors activated (19199.98 BogoMIPS) Aug 15 05:51:55 oleg133-client kernel: node 0 deferred pages initialised in 23ms Aug 15 05:51:55 oleg133-client kernel: devtmpfs: initialized Aug 15 05:51:55 oleg133-client kernel: x86/mm: Memory block size: 128MB Aug 15 05:51:55 oleg133-client kernel: gcov: version magic: 0x41383552 Aug 15 05:51:55 oleg133-client kernel: clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns Aug 15 05:51:55 oleg133-client kernel: futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: pinctrl core: initialized pinctrl subsystem Aug 15 05:51:55 oleg133-client kernel: Aug 15 05:51:55 oleg133-client kernel: ************************************************************* Aug 15 05:51:55 oleg133-client kernel: ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** Aug 15 05:51:55 oleg133-client kernel: ** ** Aug 15 05:51:55 oleg133-client kernel: ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** Aug 15 05:51:55 oleg133-client kernel: ** ** Aug 15 05:51:55 oleg133-client kernel: ** This means that this kernel is built to expose internal ** Aug 15 05:51:55 oleg133-client kernel: ** IOMMU data structures, which may compromise security on ** Aug 15 05:51:55 oleg133-client kernel: ** your system. ** Aug 15 05:51:55 oleg133-client kernel: ** ** Aug 15 05:51:55 oleg133-client kernel: ** If you see this message and you are not debugging the ** Aug 15 05:51:55 oleg133-client kernel: ** kernel, report this immediately to your vendor! ** Aug 15 05:51:55 oleg133-client kernel: ** ** Aug 15 05:51:55 oleg133-client kernel: ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** Aug 15 05:51:55 oleg133-client kernel: ************************************************************* Aug 15 05:51:55 oleg133-client kernel: NET: Registered protocol family 16 Aug 15 05:51:55 oleg133-client kernel: DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations Aug 15 05:51:55 oleg133-client kernel: DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations Aug 15 05:51:55 oleg133-client kernel: DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations Aug 15 05:51:55 oleg133-client kernel: cpuidle: using governor menu Aug 15 05:51:55 oleg133-client kernel: acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 Aug 15 05:51:55 oleg133-client kernel: PCI: Using configuration type 1 for base access Aug 15 05:51:55 oleg133-client kernel: core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on Aug 15 05:51:55 oleg133-client kernel: HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages Aug 15 05:51:55 oleg133-client kernel: HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages Aug 15 05:51:55 oleg133-client kernel: cryptd: max_cpu_qlen set to 1000 Aug 15 05:51:55 oleg133-client kernel: ACPI: Added _OSI(Module Device) Aug 15 05:51:55 oleg133-client kernel: ACPI: Added _OSI(Processor Device) Aug 15 05:51:55 oleg133-client kernel: ACPI: Added _OSI(3.0 _SCP Extensions) Aug 15 05:51:55 oleg133-client kernel: ACPI: Added _OSI(Processor Aggregator Device) Aug 15 05:51:55 oleg133-client kernel: ACPI: 1 ACPI AML tables successfully acquired and loaded Aug 15 05:51:55 oleg133-client kernel: ACPI: Interpreter enabled Aug 15 05:51:55 oleg133-client kernel: ACPI: PM: (supports S0 S3 S4 S5) Aug 15 05:51:55 oleg133-client kernel: ACPI: Using IOAPIC for interrupt routing Aug 15 05:51:55 oleg133-client kernel: PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug Aug 15 05:51:55 oleg133-client kernel: ACPI: Enabled 2 GPEs in block 00 to 0F Aug 15 05:51:55 oleg133-client kernel: ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) Aug 15 05:51:55 oleg133-client kernel: acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] Aug 15 05:51:55 oleg133-client kernel: acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] Aug 15 05:51:55 oleg133-client kernel: acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [2] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [5] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [6] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [3] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [4] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [7] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [8] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [9] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [10] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [11] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [12] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [13] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [14] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [15] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [16] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [17] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [18] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [19] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [20] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [21] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [22] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [23] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [24] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [25] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [26] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [27] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [28] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [29] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [30] registered Aug 15 05:51:55 oleg133-client kernel: acpiphp: Slot [31] registered Aug 15 05:51:55 oleg133-client kernel: PCI host bridge to bus 0000:00 Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: root bus resource [bus 00-ff] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] Aug 15 05:51:55 oleg133-client kernel: ACPI: PCI: Interrupt link LNKA configured for IRQ 10 Aug 15 05:51:55 oleg133-client kernel: ACPI: PCI: Interrupt link LNKB configured for IRQ 10 Aug 15 05:51:55 oleg133-client kernel: ACPI: PCI: Interrupt link LNKC configured for IRQ 11 Aug 15 05:51:55 oleg133-client kernel: ACPI: PCI: Interrupt link LNKD configured for IRQ 11 Aug 15 05:51:55 oleg133-client kernel: ACPI: PCI: Interrupt link LNKS configured for IRQ 9 Aug 15 05:51:55 oleg133-client kernel: iommu: Default domain type: Passthrough Aug 15 05:51:55 oleg133-client kernel: SCSI subsystem initialized Aug 15 05:51:55 oleg133-client kernel: ACPI: bus type USB registered Aug 15 05:51:55 oleg133-client kernel: usbcore: registered new interface driver usbfs Aug 15 05:51:55 oleg133-client kernel: usbcore: registered new interface driver hub Aug 15 05:51:55 oleg133-client kernel: usbcore: registered new device driver usb Aug 15 05:51:55 oleg133-client kernel: pps_core: LinuxPPS API ver. 1 registered Aug 15 05:51:55 oleg133-client kernel: pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti Aug 15 05:51:55 oleg133-client kernel: PTP clock support registered Aug 15 05:51:55 oleg133-client kernel: EDAC MC: Ver: 3.0.0 Aug 15 05:51:55 oleg133-client kernel: PCI: Using ACPI for IRQ routing Aug 15 05:51:55 oleg133-client kernel: PCI: pci_cache_line_size set to 64 bytes Aug 15 05:51:55 oleg133-client kernel: e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] Aug 15 05:51:55 oleg133-client kernel: e820: reserve RAM buffer [mem 0xbffda000-0xbfffffff] Aug 15 05:51:55 oleg133-client kernel: e820: reserve RAM buffer [mem 0x146e00000-0x147ffffff] Aug 15 05:51:55 oleg133-client kernel: NetLabel: Initializing Aug 15 05:51:55 oleg133-client kernel: NetLabel: domain hash size = 128 Aug 15 05:51:55 oleg133-client kernel: NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO Aug 15 05:51:55 oleg133-client kernel: NetLabel: unlabeled traffic allowed by default Aug 15 05:51:55 oleg133-client kernel: vgaarb: loaded Aug 15 05:51:55 oleg133-client kernel: hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 Aug 15 05:51:55 oleg133-client kernel: hpet0: 3 comparators, 64-bit 100.000000 MHz counter Aug 15 05:51:55 oleg133-client kernel: clocksource: Switched to clocksource kvm-clock Aug 15 05:51:55 oleg133-client kernel: VFS: Disk quotas dquot_6.6.0 Aug 15 05:51:55 oleg133-client kernel: VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) Aug 15 05:51:55 oleg133-client kernel: *** VALIDATE ramfs *** Aug 15 05:51:55 oleg133-client kernel: *** VALIDATE hugetlbfs *** Aug 15 05:51:55 oleg133-client kernel: pnp: PnP ACPI init Aug 15 05:51:55 oleg133-client kernel: pnp 00:00: Plug and Play ACPI device, IDs PNP0303 (active) Aug 15 05:51:55 oleg133-client kernel: pnp 00:01: Plug and Play ACPI device, IDs PNP0f13 (active) Aug 15 05:51:55 oleg133-client kernel: pnp 00:02: [dma 2] Aug 15 05:51:55 oleg133-client kernel: pnp 00:02: Plug and Play ACPI device, IDs PNP0700 (active) Aug 15 05:51:55 oleg133-client kernel: pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) Aug 15 05:51:55 oleg133-client kernel: pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) Aug 15 05:51:55 oleg133-client kernel: pnp 00:05: Plug and Play ACPI device, IDs PNP0b00 (active) Aug 15 05:51:55 oleg133-client kernel: pnp: PnP ACPI: found 6 devices Aug 15 05:51:55 oleg133-client kernel: clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] Aug 15 05:51:55 oleg133-client kernel: pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] Aug 15 05:51:55 oleg133-client kernel: NET: Registered protocol family 2 Aug 15 05:51:55 oleg133-client kernel: IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: TCP: Hash tables configured (established 65536 bind 65536) Aug 15 05:51:55 oleg133-client kernel: MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) Aug 15 05:51:55 oleg133-client kernel: NET: Registered protocol family 1 Aug 15 05:51:55 oleg133-client kernel: RPC: Registered named UNIX socket transport module. Aug 15 05:51:55 oleg133-client kernel: RPC: Registered udp transport module. Aug 15 05:51:55 oleg133-client kernel: RPC: Registered tcp transport module. Aug 15 05:51:55 oleg133-client kernel: RPC: Registered tcp NFSv4.1 backchannel transport module. Aug 15 05:51:55 oleg133-client kernel: NET: Registered protocol family 44 Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:00.0: Limiting direct PCI/PCI transfers Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.0: PIIX3: Enabling Passive Release Aug 15 05:51:55 oleg133-client kernel: pci 0000:00:01.0: Activating ISA DMA hang workarounds Aug 15 05:51:55 oleg133-client kernel: PCI: CLS 0 bytes, default 64 Aug 15 05:51:55 oleg133-client kernel: Unpacking initramfs... Aug 15 05:51:55 oleg133-client kernel: debug: unmapping init [mem 0xffff9936fcc64000-0xffff9936fffcffff] Aug 15 05:51:55 oleg133-client kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB) Aug 15 05:51:55 oleg133-client kernel: software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) Aug 15 05:51:55 oleg133-client kernel: clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns Aug 15 05:51:55 oleg133-client kernel: Initialise system trusted keyrings Aug 15 05:51:55 oleg133-client kernel: Key type blacklist registered Aug 15 05:51:55 oleg133-client kernel: workingset: timestamp_bits=36 max_order=20 bucket_order=0 Aug 15 05:51:55 oleg133-client kernel: zbud: loaded Aug 15 05:51:55 oleg133-client kernel: *** VALIDATE nfs *** Aug 15 05:51:55 oleg133-client kernel: *** VALIDATE nfs4 *** Aug 15 05:51:55 oleg133-client kernel: pstore: using deflate compression Aug 15 05:51:55 oleg133-client kernel: Platform Keyring initialized Aug 15 05:51:55 oleg133-client kernel: NET: Registered protocol family 38 Aug 15 05:51:55 oleg133-client kernel: Key type asymmetric registered Aug 15 05:51:55 oleg133-client kernel: Asymmetric key parser 'x509' registered Aug 15 05:51:55 oleg133-client kernel: Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) Aug 15 05:51:55 oleg133-client kernel: io scheduler mq-deadline registered Aug 15 05:51:55 oleg133-client kernel: io scheduler kyber registered Aug 15 05:51:55 oleg133-client kernel: io scheduler bfq registered Aug 15 05:51:55 oleg133-client kernel: atomic64_test: passed for x86-64 platform with CX8 and with SSE Aug 15 05:51:55 oleg133-client kernel: shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 Aug 15 05:51:55 oleg133-client kernel: input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 Aug 15 05:51:55 oleg133-client kernel: ACPI: Power Button [PWRF] Aug 15 05:51:55 oleg133-client kernel: ACPI: \_SB_.LNKB: Enabled at IRQ 10 Aug 15 05:51:55 oleg133-client kernel: ACPI: \_SB_.LNKA: Enabled at IRQ 11 Aug 15 05:51:55 oleg133-client kernel: Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled Aug 15 05:51:55 oleg133-client kernel: 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A Aug 15 05:51:55 oleg133-client kernel: 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A Aug 15 05:51:55 oleg133-client kernel: Non-volatile memory driver v1.3 Aug 15 05:51:55 oleg133-client kernel: Linux agpgart interface v0.103 Aug 15 05:51:55 oleg133-client kernel: virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) Aug 15 05:51:55 oleg133-client kernel: vda: detected capacity change from 0 to 74702848 Aug 15 05:51:55 oleg133-client kernel: virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) Aug 15 05:51:55 oleg133-client kernel: vdb: detected capacity change from 0 to 1073741824 Aug 15 05:51:55 oleg133-client kernel: libphy: Fixed MDIO Bus: probed Aug 15 05:51:55 oleg133-client kernel: usbcore: registered new interface driver usbserial_generic Aug 15 05:51:55 oleg133-client kernel: usbserial: USB Serial support registered for generic Aug 15 05:51:55 oleg133-client kernel: i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 Aug 15 05:51:55 oleg133-client kernel: serio: i8042 KBD port at 0x60,0x64 irq 1 Aug 15 05:51:55 oleg133-client kernel: serio: i8042 AUX port at 0x60,0x64 irq 12 Aug 15 05:51:55 oleg133-client kernel: mousedev: PS/2 mouse device common for all mice Aug 15 05:51:55 oleg133-client kernel: rtc_cmos 00:05: RTC can wake from S4 Aug 15 05:51:55 oleg133-client kernel: rtc_cmos 00:05: registered as rtc0 Aug 15 05:51:55 oleg133-client kernel: rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs Aug 15 05:51:55 oleg133-client kernel: intel_pstate: CPU model not supported Aug 15 05:51:55 oleg133-client kernel: input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 Aug 15 05:51:55 oleg133-client kernel: hid: raw HID events driver (C) Jiri Kosina Aug 15 05:51:55 oleg133-client kernel: input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 Aug 15 05:51:55 oleg133-client kernel: usbcore: registered new interface driver usbhid Aug 15 05:51:55 oleg133-client kernel: input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 Aug 15 05:51:55 oleg133-client kernel: usbhid: USB HID core driver Aug 15 05:51:55 oleg133-client kernel: drop_monitor: Initializing network drop monitor service Aug 15 05:51:55 oleg133-client kernel: Initializing XFRM netlink socket Aug 15 05:51:55 oleg133-client kernel: NET: Registered protocol family 10 Aug 15 05:51:55 oleg133-client kernel: Segment Routing with IPv6 Aug 15 05:51:55 oleg133-client kernel: NET: Registered protocol family 17 Aug 15 05:51:55 oleg133-client kernel: mpls_gso: MPLS GSO support Aug 15 05:51:55 oleg133-client kernel: start plist test Aug 15 05:51:55 oleg133-client kernel: end plist test Aug 15 05:51:55 oleg133-client kernel: RAS: Correctable Errors collector initialized. Aug 15 05:51:55 oleg133-client kernel: AVX version of gcm_enc/dec engaged. Aug 15 05:51:55 oleg133-client kernel: AES CTR mode by8 optimization enabled Aug 15 05:51:55 oleg133-client kernel: sched_clock: Marking stable (2948816082, 0)->(3938959853, -990143771) Aug 15 05:51:55 oleg133-client kernel: registered taskstats version 1 Aug 15 05:51:55 oleg133-client kernel: Loading compiled-in X.509 certificates Aug 15 05:51:55 oleg133-client kernel: zswap: loaded using pool lzo/zbud Aug 15 05:51:55 oleg133-client kernel: Key type big_key registered Aug 15 05:51:55 oleg133-client kernel: Key type encrypted registered Aug 15 05:51:55 oleg133-client kernel: ima: No TPM chip found, activating TPM-bypass! Aug 15 05:51:55 oleg133-client kernel: ima: Allocated hash algorithm: sha1 Aug 15 05:51:55 oleg133-client kernel: ima: No architecture policies found Aug 15 05:51:55 oleg133-client kernel: evm: Initialising EVM extended attributes: Aug 15 05:51:55 oleg133-client kernel: evm: security.selinux Aug 15 05:51:55 oleg133-client kernel: evm: security.ima Aug 15 05:51:55 oleg133-client kernel: evm: security.capability Aug 15 05:51:55 oleg133-client kernel: evm: HMAC attrs: 0x1 Aug 15 05:51:55 oleg133-client kernel: rtc_cmos 00:05: setting system clock to 2026-08-15 09:51:55 UTC (1786787515) Aug 15 05:51:55 oleg133-client kernel: debug: unmapping init [mem 0xffffffff9d203000-0xffffffff9d3fffff] Aug 15 05:51:55 oleg133-client kernel: debug: unmapping init [mem 0xffffffff9bf82000-0xffffffff9c258fff] Aug 15 05:51:55 oleg133-client kernel: Write protecting the kernel read-only data: 28672k Aug 15 05:51:55 oleg133-client kernel: debug: unmapping init [mem 0xffffffff9a603000-0xffffffff9a7fffff] Aug 15 05:51:55 oleg133-client kernel: debug: unmapping init [mem 0xffffffff9af14000-0xffffffff9affffff] Aug 15 05:51:55 oleg133-client systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) Aug 15 05:51:55 oleg133-client systemd[1]: Detected virtualization kvm. Aug 15 05:51:55 oleg133-client systemd[1]: Detected architecture x86-64. Aug 15 05:51:55 oleg133-client systemd[1]: Running in initial RAM disk. Aug 15 05:51:55 oleg133-client systemd[1]: No hostname configured. Aug 15 05:51:55 oleg133-client systemd[1]: Set hostname to . Aug 15 05:51:55 oleg133-client kernel: random: systemd: uninitialized urandom read (16 bytes read) Aug 15 05:51:55 oleg133-client systemd[1]: Initializing machine ID from random generator. Aug 15 05:51:55 oleg133-client kernel: random: ln: uninitialized urandom read (6 bytes read) Aug 15 05:51:55 oleg133-client kernel: random: systemd: uninitialized urandom read (16 bytes read) Aug 15 05:51:55 oleg133-client systemd[1]: Listening on udev Control Socket. Aug 15 05:51:55 oleg133-client systemd[1]: Listening on udev Kernel Socket. Aug 15 05:51:55 oleg133-client systemd[1]: Reached target Initrd Root Device. Aug 15 05:51:55 oleg133-client systemd-journald[235]: Journal started Aug 15 05:51:55 oleg133-client systemd-journald[235]: Runtime journal (/run/log/journal/9433db0efa134754ad387e3ebf090890) is 8.0M, max 184.6M, 176.6M free. Aug 15 05:51:55 oleg133-client systemd-vconsole-setup[231]: KD_FONT_OP_GET failed while trying to get the font metadata: Invalid argument Aug 15 05:51:55 oleg133-client systemd-vconsole-setup[231]: Fonts will not be copied to remaining consoles Aug 15 05:51:55 oleg133-client dracut-cmdline[256]: dracut-8.10 (Green Obsidian) dracut-049-233.git20240115.el8 Aug 15 05:51:55 oleg133-client dracut-cmdline[256]: Using kernel command line parameters: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 Aug 15 05:51:56 oleg133-client systemd[1]: Started dracut cmdline hook. Aug 15 05:51:56 oleg133-client systemd[1]: Starting dracut pre-udev hook... Aug 15 05:51:56 oleg133-client kernel: device-mapper: uevent: version 1.0.3 Aug 15 05:51:56 oleg133-client kernel: device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com Aug 15 05:51:56 oleg133-client rpc.statd[388]: Version 2.3.3 starting Aug 15 05:51:56 oleg133-client rpc.statd[388]: Initializing NSM state Aug 15 05:51:56 oleg133-client rpc.idmapd[393]: Setting log level to 0 Aug 15 05:51:56 oleg133-client systemd[1]: Started dracut pre-udev hook. Aug 15 05:51:56 oleg133-client systemd[1]: Starting udev Kernel Device Manager... Aug 15 05:51:56 oleg133-client systemd[1]: Started udev Kernel Device Manager. Aug 15 05:51:56 oleg133-client systemd[1]: Starting dracut pre-trigger hook... Aug 15 05:51:56 oleg133-client systemd[1]: Started dracut pre-trigger hook. Aug 15 05:51:56 oleg133-client systemd[1]: Starting udev Coldplug all Devices... Aug 15 05:51:56 oleg133-client systemd[1]: Mounting Kernel Configuration File System... Aug 15 05:51:56 oleg133-client systemd[1]: Mounted Kernel Configuration File System. Aug 15 05:51:57 oleg133-client systemd[1]: Started udev Coldplug all Devices. Aug 15 05:51:57 oleg133-client systemd[1]: Reached target System Initialization. Aug 15 05:51:57 oleg133-client systemd[1]: Reached target Basic System. Aug 15 05:51:57 oleg133-client systemd[1]: Started Hardware RNG Entropy Gatherer Daemon. Aug 15 05:51:57 oleg133-client systemd[1]: Starting dracut initqueue hook... Aug 15 05:51:57 oleg133-client rngd[444]: Disabling 7: PKCS11 Entropy generator (pkcs11) Aug 15 05:51:57 oleg133-client rngd[444]: Disabling 5: NIST Network Entropy Beacon (nist) Aug 15 05:51:57 oleg133-client rngd[444]: Disabling 9: Qrypt quantum entropy beacon (qrypt) Aug 15 05:51:57 oleg133-client rngd[444]: Initializing available sources Aug 15 05:51:57 oleg133-client rngd[444]: [hwrng ]: Initialization Failed Aug 15 05:51:57 oleg133-client rngd[444]: [rdrand]: Enabling RDRAND rng support Aug 15 05:51:57 oleg133-client rngd[444]: [rdrand]: Initialized Aug 15 05:51:57 oleg133-client rngd[444]: [jitter]: JITTER timeout set to 5 sec Aug 15 05:51:57 oleg133-client systemd-udevd[447]: Using default interface naming scheme 'rhel-8.0'. Aug 15 05:51:57 oleg133-client systemd-udevd[447]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Aug 15 05:51:57 oleg133-client kernel: virtio_net virtio0 ens2: renamed from eth0 Aug 15 05:51:57 oleg133-client kernel: libata version 3.00 loaded. Aug 15 05:51:57 oleg133-client kernel: ata_piix 0000:00:01.1: version 2.13 Aug 15 05:51:57 oleg133-client kernel: scsi host0: ata_piix Aug 15 05:51:57 oleg133-client kernel: scsi host1: ata_piix Aug 15 05:51:57 oleg133-client kernel: ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 Aug 15 05:51:57 oleg133-client kernel: ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 Aug 15 05:51:57 oleg133-client rngd[444]: [jitter]: Initializing AES buffer Aug 15 05:51:57 oleg133-client systemd-udevd[446]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Aug 15 05:51:57 oleg133-client dracut-initqueue[500]: Starting dhcp for interface ens2 Aug 15 05:51:58 oleg133-client dracut-initqueue[538]: dhcp: PREINIT ens2 up Aug 15 05:51:58 oleg133-client dhclient[537]: Created duid "\000\004\2243\333\016\372\023GT\2558~>\277\011\010\220". Aug 15 05:51:58 oleg133-client dhclient[537]: DHCPDISCOVER on ens2 to 255.255.255.255 port 67 interval 6 (xid=0xd0572f76) Aug 15 05:51:58 oleg133-client dhclient[537]: DHCPREQUEST on ens2 to 255.255.255.255 port 67 (xid=0xd0572f76) Aug 15 05:51:58 oleg133-client dhclient[537]: DHCPOFFER from 192.168.201.254 Aug 15 05:51:58 oleg133-client dhclient[537]: DHCPACK from 192.168.201.254 (xid=0xd0572f76) Aug 15 05:51:58 oleg133-client dracut-initqueue[556]: dhcp: BOND setting ens2 Aug 15 05:52:00 oleg133-client dhclient[537]: bound to 192.168.201.33 -- renewal in 27218 seconds. Aug 15 05:52:01 oleg133-client dracut-initqueue[591]: RTNETLINK answers: File exists Aug 15 05:52:01 oleg133-client kernel: random: fast init done Aug 15 05:52:01 oleg133-client systemd[1]: Reloading. Aug 15 05:52:01 oleg133-client systemd[619]: /usr/lib/systemd/system-generators/systemd-fstab-generator failed with exit status 1. Aug 15 05:52:02 oleg133-client rngd[444]: [jitter]: Unable to obtain AES key, disabling JITTER source Aug 15 05:52:02 oleg133-client rngd[444]: [jitter]: Initialization Failed Aug 15 05:52:02 oleg133-client kernel: random: crng init done Aug 15 05:52:02 oleg133-client kernel: random: 7 urandom warning(s) missed due to ratelimiting Aug 15 05:52:02 oleg133-client systemd[1]: Starting nbd nbd0... Aug 15 05:52:02 oleg133-client nbd-client[654]: Negotiation: ..size = 30720MB Aug 15 05:52:02 oleg133-client nbd-client[654]: Connected /dev/nbd0 Aug 15 05:52:02 oleg133-client systemd[1]: nbd0.service: Succeeded. Aug 15 05:52:02 oleg133-client systemd[1]: Started nbd nbd0. Aug 15 05:52:02 oleg133-client systemd[1]: Started dracut initqueue hook. Aug 15 05:52:02 oleg133-client systemd[1]: Mounting /sysroot... Aug 15 05:52:02 oleg133-client systemd[1]: Reached target Remote File Systems (Pre). Aug 15 05:52:02 oleg133-client systemd[1]: Reached target Remote File Systems. Aug 15 05:52:02 oleg133-client kernel: EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) Aug 15 05:52:02 oleg133-client systemd[1]: Mounted /sysroot. Aug 15 05:52:02 oleg133-client systemd[1]: Reached target Initrd Root File System. Aug 15 05:52:02 oleg133-client systemd[1]: Starting Reload Configuration from the Real Root... Aug 15 05:52:02 oleg133-client systemd[1]: Reloading. Aug 15 05:52:02 oleg133-client systemd[678]: /usr/lib/systemd/system-generators/systemd-fstab-generator failed with exit status 1. Aug 15 05:52:02 oleg133-client systemd[1]: initrd-parse-etc.service: Succeeded. Aug 15 05:52:02 oleg133-client systemd[1]: Started Reload Configuration from the Real Root. Aug 15 05:52:02 oleg133-client systemd[1]: Reached target Initrd File Systems. Aug 15 05:52:02 oleg133-client systemd[1]: Reached target Initrd Default Target. Aug 15 05:52:02 oleg133-client systemd[1]: Starting dracut pre-pivot and cleanup hook... Aug 15 05:52:02 oleg133-client systemd[1]: var-lib-nfs-rpc_pipefs.mount: Succeeded. Aug 15 05:52:02 oleg133-client systemd[1]: Started dracut pre-pivot and cleanup hook. Aug 15 05:52:02 oleg133-client systemd[1]: Starting Cleaning Up and Shutting Down Daemons... Aug 15 05:52:02 oleg133-client rngd[444]: [rdrand]: Shutting down Aug 15 05:52:02 oleg133-client systemd[1]: Stopping Hardware RNG Entropy Gatherer Daemon... Aug 15 05:52:02 oleg133-client systemd[1]: Stopped target Timers. Aug 15 05:52:02 oleg133-client systemd[1]: dracut-pre-pivot.service: Succeeded. Aug 15 05:52:02 oleg133-client systemd[1]: Stopped dracut pre-pivot and cleanup hook. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Initrd Default Target. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Initrd Root Device. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Remote File Systems. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Remote File Systems (Pre). Aug 15 05:52:03 oleg133-client systemd[1]: dracut-initqueue.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped dracut initqueue hook. Aug 15 05:52:03 oleg133-client systemd[1]: rngd.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped Hardware RNG Entropy Gatherer Daemon. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Basic System. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Slices. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Paths. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Sockets. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target System Initialization. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Local Encrypted Volumes. Aug 15 05:52:03 oleg133-client systemd[1]: systemd-ask-password-console.path: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. Aug 15 05:52:03 oleg133-client systemd[1]: systemd-sysctl.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped Apply Kernel Variables. Aug 15 05:52:03 oleg133-client systemd[1]: systemd-udev-trigger.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped udev Coldplug all Devices. Aug 15 05:52:03 oleg133-client systemd[1]: dracut-pre-trigger.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped dracut pre-trigger hook. Aug 15 05:52:03 oleg133-client systemd[1]: systemd-tmpfiles-setup.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped Create Volatile Files and Directories. Aug 15 05:52:03 oleg133-client systemd[1]: Stopping udev Kernel Device Manager... Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Local File Systems. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped target Swap. Aug 15 05:52:03 oleg133-client systemd[1]: initrd-cleanup.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Started Cleaning Up and Shutting Down Daemons. Aug 15 05:52:03 oleg133-client systemd[1]: systemd-udevd.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped udev Kernel Device Manager. Aug 15 05:52:03 oleg133-client systemd[1]: dracut-pre-udev.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped dracut pre-udev hook. Aug 15 05:52:03 oleg133-client systemd[1]: dracut-cmdline.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped dracut cmdline hook. Aug 15 05:52:03 oleg133-client systemd[1]: systemd-tmpfiles-setup-dev.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped Create Static Device Nodes in /dev. Aug 15 05:52:03 oleg133-client systemd[1]: kmod-static-nodes.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Stopped Create list of required static device nodes for the current kernel. Aug 15 05:52:03 oleg133-client systemd[1]: systemd-udevd-kernel.socket: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Closed udev Kernel Socket. Aug 15 05:52:03 oleg133-client systemd[1]: systemd-udevd-control.socket: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Closed udev Control Socket. Aug 15 05:52:03 oleg133-client systemd[1]: Starting Cleanup udevd DB... Aug 15 05:52:03 oleg133-client systemd[1]: initrd-udevadm-cleanup-db.service: Succeeded. Aug 15 05:52:03 oleg133-client systemd[1]: Started Cleanup udevd DB. Aug 15 05:52:03 oleg133-client systemd[1]: Reached target Switch Root. Aug 15 05:52:03 oleg133-client systemd[1]: Starting Switch Root... Aug 15 05:52:03 oleg133-client systemd[1]: Switching root. Aug 15 05:52:03 oleg133-client systemd-journald[235]: Journal stopped Aug 15 05:52:04 oleg133-client kernel: printk: systemd: 26 output lines suppressed due to ratelimiting Aug 15 05:52:04 oleg133-client kernel: SELinux: Disabled at runtime. Aug 15 05:52:04 oleg133-client systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) Aug 15 05:52:04 oleg133-client systemd[1]: Detected virtualization kvm. Aug 15 05:52:04 oleg133-client systemd[1]: Detected architecture x86-64. Aug 15 05:52:04 oleg133-client systemd[1]: initrd-switch-root.service: Succeeded. Aug 15 05:52:04 oleg133-client systemd[1]: Stopped Switch Root. Aug 15 05:52:04 oleg133-client systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. Aug 15 05:52:04 oleg133-client systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. Aug 15 05:52:04 oleg133-client systemd[1]: Stopped Journal Service. Aug 15 05:52:04 oleg133-client systemd[1]: Starting Journal Service... Aug 15 05:52:04 oleg133-client systemd[1]: Starting Create list of required static device nodes for the current kernel... Aug 15 05:52:04 oleg133-client kernel: Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Aug 15 05:52:04 oleg133-client systemd-journald[794]: Journal started Aug 15 05:52:04 oleg133-client systemd-journald[794]: Runtime journal (/run/log/journal/a82c5ce2bafb4275b974cc7d83c03742) is 1.2M, max 10.0M, 8.7M free. Aug 15 05:52:04 oleg133-client systemd[1]: systemd-journald.service: Succeeded. Aug 15 05:52:04 oleg133-client systemd-remount-fs[798]: /usr/bin/mount for / exited with exit status 32. Aug 15 05:52:04 oleg133-client systemd-remount-fs[802]: mount.nfs: access denied by server while mounting 192.168.10.1:/exports/rhel8.5-nfsroot Aug 15 05:52:04 oleg133-client systemd[1]: Started Create list of required static device nodes for the current kernel. Aug 15 05:52:04 oleg133-client systemd[1]: Mounted POSIX Message Queue File System. Aug 15 05:52:04 oleg133-client systemd[1]: systemd-remount-fs.service: Main process exited, code=exited, status=1/FAILURE Aug 15 05:52:04 oleg133-client systemd[1]: systemd-remount-fs.service: Failed with result 'exit-code'. Aug 15 05:52:04 oleg133-client systemd[1]: Failed to start Remount Root and Kernel File Systems. Aug 15 05:52:04 oleg133-client systemd[1]: Activated swap /dev/disk/by-label/SWAP. Aug 15 05:52:04 oleg133-client systemd[1]: Mounted Kernel Debug File System. Aug 15 05:52:04 oleg133-client systemd[1]: Mounted Huge Pages File System. Aug 15 05:52:04 oleg133-client systemd[1]: Started Apply Kernel Variables. Aug 15 05:52:04 oleg133-client systemd[1]: Reached target Swap. Aug 15 05:52:04 oleg133-client systemd[1]: Starting Configure read-only root support... Aug 15 05:52:04 oleg133-client systemd[1]: Starting Create Static Device Nodes in /dev... Aug 15 05:52:04 oleg133-client systemd[1]: Starting Flush Journal to Persistent Storage... Aug 15 05:52:04 oleg133-client systemd-journald[794]: Runtime journal (/run/log/journal/a82c5ce2bafb4275b974cc7d83c03742) is 1.2M, max 10.0M, 8.7M free. Aug 15 05:52:04 oleg133-client systemd[1]: Started Flush Journal to Persistent Storage. Aug 15 05:52:04 oleg133-client systemd[1]: Started Create Static Device Nodes in /dev. Aug 15 05:52:04 oleg133-client systemd[1]: Starting udev Kernel Device Manager... Aug 15 05:52:04 oleg133-client systemd[1]: Reached target Local File Systems (Pre). Aug 15 05:52:04 oleg133-client systemd[1]: Mounting /mnt... Aug 15 05:52:04 oleg133-client systemd[1]: Mounting /home/green/git/lustre-release... Aug 15 05:52:05 oleg133-client systemd[1]: Started udev Coldplug all Devices. Aug 15 05:52:05 oleg133-client systemd[1]: Mounted /mnt. Aug 15 05:52:05 oleg133-client kernel: squashfs: version 4.0 (2009/01/31) Phillip Lougher Aug 15 05:52:05 oleg133-client systemd[1]: Mounted /home/green/git/lustre-release. Aug 15 05:52:05 oleg133-client systemd[1]: Started udev Kernel Device Manager. Aug 15 05:52:05 oleg133-client kernel: piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 Aug 15 05:52:05 oleg133-client kernel: input: PC Speaker as /devices/platform/pcspkr/input/input5 Aug 15 05:52:05 oleg133-client kernel: RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer Aug 15 05:52:05 oleg133-client kernel: EDAC sbridge: Seeking for: PCI ID 8086:0ea0 Aug 15 05:52:05 oleg133-client kernel: EDAC sbridge: Ver: 1.1.2 Aug 15 05:52:05 oleg133-client systemd-udevd[885]: Using default interface naming scheme 'rhel-8.0'. Aug 15 05:52:05 oleg133-client systemd-udevd[885]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Aug 15 05:52:05 oleg133-client systemd-udevd[867]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. Aug 15 05:52:06 oleg133-client kernel: Key type dns_resolver registered Aug 15 05:52:06 oleg133-client kernel: NFS: Registering the id_resolver key type Aug 15 05:52:06 oleg133-client kernel: Key type id_resolver registered Aug 15 05:52:06 oleg133-client kernel: Key type id_legacy registered Aug 15 05:52:06 oleg133-client nfsrahead[1002]: setting /var/lib/stateless/state readahead to 128 Aug 15 05:52:06 oleg133-client systemd[1]: Started Configure read-only root support. Aug 15 05:52:06 oleg133-client systemd[1]: Starting Load/Save Random Seed... Aug 15 05:52:07 oleg133-client systemd[1]: Reached target Local File Systems. Aug 15 05:52:07 oleg133-client systemd[1]: Starting Rebuild Dynamic Linker Cache... Aug 15 05:52:07 oleg133-client systemd[1]: Starting Create Volatile Files and Directories... Aug 15 05:52:07 oleg133-client systemd[1]: Starting Mark the need to relabel after reboot... Aug 15 05:52:07 oleg133-client touch[1017]: /bin/touch: cannot touch '/.autorelabel': Read-only file system Aug 15 05:52:07 oleg133-client systemd[1]: Started Load/Save Random Seed. Aug 15 05:52:07 oleg133-client systemd[1]: Started Mark the need to relabel after reboot. Aug 15 05:52:07 oleg133-client systemd-tmpfiles[1015]: rm(/var/lib/rpm/__db.003): Read-only file system Aug 15 05:52:07 oleg133-client systemd-tmpfiles[1015]: rm(/var/lib/rpm/__db.002): Read-only file system Aug 15 05:52:07 oleg133-client systemd-tmpfiles[1015]: rm(/var/lib/rpm/__db.001): Read-only file system Aug 15 05:52:07 oleg133-client systemd[1]: Started Create Volatile Files and Directories. Aug 15 05:52:07 oleg133-client systemd[1]: Starting Update UTMP about System Boot/Shutdown... Aug 15 05:52:07 oleg133-client systemd[1]: Starting RPC Bind... Aug 15 05:52:07 oleg133-client systemd[1]: Started Update UTMP about System Boot/Shutdown. Aug 15 05:52:07 oleg133-client systemd[1]: Started RPC Bind. Aug 15 05:52:07 oleg133-client systemd[1]: Started Rebuild Dynamic Linker Cache. Aug 15 05:52:07 oleg133-client systemd[1]: Starting Update is Completed... Aug 15 05:52:07 oleg133-client systemd[1]: Started Update is Completed. Aug 15 05:52:07 oleg133-client systemd[1]: Reached target System Initialization. Aug 15 05:52:07 oleg133-client systemd[1]: Started Daily Cleanup of Temporary Directories. Aug 15 05:52:07 oleg133-client systemd[1]: Started dnf makecache --timer. Aug 15 05:52:07 oleg133-client systemd[1]: Started daily update of the root trust anchor for DNSSEC. Aug 15 05:52:07 oleg133-client systemd[1]: Reached target Timers. Aug 15 05:52:07 oleg133-client systemd[1]: Listening on D-Bus System Message Bus Socket. Aug 15 05:52:07 oleg133-client systemd[1]: Reached target Sockets. Aug 15 05:52:07 oleg133-client systemd[1]: Reached target Basic System. Aug 15 05:52:07 oleg133-client systemd[1]: Starting Restore /run/initramfs on shutdown... Aug 15 05:52:07 oleg133-client systemd[1]: Started Hardware RNG Entropy Gatherer Daemon. Aug 15 05:52:07 oleg133-client systemd[1]: Started D-Bus System Message Bus. Aug 15 05:52:07 oleg133-client systemd[1]: Starting Network Manager... Aug 15 05:52:07 oleg133-client systemd[1]: Reached target sshd-keygen.target. Aug 15 05:52:07 oleg133-client systemd[1]: Started irqbalance daemon. Aug 15 05:52:07 oleg133-client rngd[1030]: Disabling 7: PKCS11 Entropy generator (pkcs11) Aug 15 05:52:07 oleg133-client rngd[1030]: Disabling 5: NIST Network Entropy Beacon (nist) Aug 15 05:52:07 oleg133-client rngd[1030]: Disabling 9: Qrypt quantum entropy beacon (qrypt) Aug 15 05:52:07 oleg133-client rngd[1030]: Initializing available sources Aug 15 05:52:07 oleg133-client rngd[1030]: [hwrng ]: Initialization Failed Aug 15 05:52:07 oleg133-client rngd[1030]: [rdrand]: Enabling RDRAND rng support Aug 15 05:52:07 oleg133-client rngd[1030]: [rdrand]: Initialized Aug 15 05:52:07 oleg133-client rngd[1030]: [jitter]: JITTER timeout set to 5 sec Aug 15 05:52:08 oleg133-client systemd[1]: Starting Login Service... Aug 15 05:52:08 oleg133-client systemd[1]: Started Restore /run/initramfs on shutdown. Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.0855] NetworkManager (version 1.40.16-19.el8_10) is starting... (boot:e710a0ee-b973-4105-8d3b-8a68ee3b0f4c) Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.0883] Read config: /etc/NetworkManager/NetworkManager.conf (run: 10-dracut-dhclient.conf) Aug 15 05:52:08 oleg133-client systemd-logind[1034]: Watching system buttons on /dev/input/event0 (Power Button) Aug 15 05:52:08 oleg133-client systemd-logind[1034]: Watching system buttons on /dev/input/event1 (AT Translated Set 2 keyboard) Aug 15 05:52:08 oleg133-client systemd-logind[1034]: New seat seat0. Aug 15 05:52:08 oleg133-client systemd[1]: Started Login Service. Aug 15 05:52:08 oleg133-client systemd[1]: Started Network Manager. Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.1139] bus-manager: acquired D-Bus service "org.freedesktop.NetworkManager" Aug 15 05:52:08 oleg133-client systemd[1]: Starting Network Manager Wait Online... Aug 15 05:52:08 oleg133-client systemd[1]: Reached target Network. Aug 15 05:52:08 oleg133-client systemd[1]: Starting GSSAPI Proxy Daemon... Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.1360] manager[0x55e8d6115020]: monitoring kernel firmware directory '/lib/firmware'. Aug 15 05:52:08 oleg133-client dbus-daemon[1031]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.2' (uid=0 pid=1032 comm="/usr/sbin/NetworkManager --no-daemon ") Aug 15 05:52:08 oleg133-client systemd[1]: Starting Dynamic System Tuning Daemon... Aug 15 05:52:08 oleg133-client rngd[1030]: [jitter]: Initializing AES buffer Aug 15 05:52:08 oleg133-client systemd[1]: Starting OpenSSH server daemon... Aug 15 05:52:08 oleg133-client systemd[1]: Started GSSAPI Proxy Daemon. Aug 15 05:52:08 oleg133-client systemd[1]: Reached target NFS client services. Aug 15 05:52:08 oleg133-client systemd[1]: Reached target Remote File Systems (Pre). Aug 15 05:52:08 oleg133-client systemd[1]: Reached target Remote File Systems. Aug 15 05:52:08 oleg133-client systemd[1]: Starting Permit User Sessions... Aug 15 05:52:08 oleg133-client systemd[1]: Starting Hostname Service... Aug 15 05:52:08 oleg133-client sshd[1054]: Server listening on 0.0.0.0 port 22. Aug 15 05:52:08 oleg133-client sshd[1054]: Server listening on :: port 22. Aug 15 05:52:08 oleg133-client systemd[1]: Started OpenSSH server daemon. Aug 15 05:52:08 oleg133-client systemd[1]: Started Permit User Sessions. Aug 15 05:52:08 oleg133-client systemd[1]: Started Getty on tty1. Aug 15 05:52:08 oleg133-client systemd[1]: Started Command Scheduler. Aug 15 05:52:08 oleg133-client systemd[1]: Started Serial Getty on ttyS1. Aug 15 05:52:08 oleg133-client crond[1070]: (CRON) STARTUP (1.5.2) Aug 15 05:52:08 oleg133-client systemd[1]: Started Serial Getty on ttyS0. Aug 15 05:52:08 oleg133-client crond[1070]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 46% if used.) Aug 15 05:52:08 oleg133-client crond[1070]: (CRON) INFO (running with inotify support) Aug 15 05:52:08 oleg133-client systemd[1]: Reached target Login Prompts. Aug 15 05:52:08 oleg133-client dbus-daemon[1031]: [system] Successfully activated service 'org.freedesktop.hostname1' Aug 15 05:52:08 oleg133-client systemd[1]: Started Hostname Service. Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6282] hostname: hostname: using hostnamed Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6290] dns-mgr: init: dns=default,systemd-resolved rc-manager=symlink Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6443] Loaded device plugin: NMTeamFactory (/usr/lib64/NetworkManager/1.40.16-19.el8_10/libnm-device-plugin-team.so) Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6444] manager: rfkill: Wi-Fi enabled by radio killswitch; enabled by state file Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6445] manager: rfkill: WWAN enabled by radio killswitch; enabled by state file Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6446] manager: Networking is enabled by state file Aug 15 05:52:08 oleg133-client dbus-daemon[1031]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.2' (uid=0 pid=1032 comm="/usr/sbin/NetworkManager --no-daemon ") Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6568] settings: Loaded settings plugin: ifcfg-rh ("/usr/lib64/NetworkManager/1.40.16-19.el8_10/libnm-settings-plugin-ifcfg-rh.so") Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6572] settings: Loaded settings plugin: keyfile (internal) Aug 15 05:52:08 oleg133-client systemd[1]: Starting Network Manager Script Dispatcher Service... Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6747] dhcp: init: Using DHCP client 'dhclient' Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6752] device (lo): carrier: link connected Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6759] manager: (lo): new Generic device (/org/freedesktop/NetworkManager/Devices/1) Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6781] device (ens2): carrier: link connected Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6801] manager: (ens2): new Ethernet device (/org/freedesktop/NetworkManager/Devices/2) Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6815] manager: (ens2): assume: will attempt to assume matching connection 'ens2' (60d4720e-3882-4c64-bd81-73f2a7b6c38b) (guessed) Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6819] manager: (ens2): assume: taking over an initramfs-configured connection Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6821] device (ens2): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'assume') Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6833] device (ens2): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'assume') Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.6887] device (ens2): Activation: starting connection 'ens2' (60d4720e-3882-4c64-bd81-73f2a7b6c38b) Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.7023] device (ens2): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'assume') Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.7032] device (ens2): state change: prepare -> config (reason 'none', sys-iface-state: 'assume') Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.7038] device (ens2): state change: config -> ip-config (reason 'none', sys-iface-state: 'assume') Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.7061] dhcp4 (ens2): activation: beginning transaction (timeout in 45 seconds) Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.7220] dhcp4 (ens2): dhclient started with pid 1074 Aug 15 05:52:08 oleg133-client dbus-daemon[1031]: [system] Activating via systemd: service name='org.freedesktop.resolve1' unit='dbus-org.freedesktop.resolve1.service' requested by ':1.2' (uid=0 pid=1032 comm="/usr/sbin/NetworkManager --no-daemon ") Aug 15 05:52:08 oleg133-client dbus-daemon[1031]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.resolve1.service': Unit dbus-org.freedesktop.resolve1.service not found. Aug 15 05:52:08 oleg133-client dbus-daemon[1031]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher' Aug 15 05:52:08 oleg133-client systemd[1]: Started Network Manager Script Dispatcher Service. Aug 15 05:52:08 oleg133-client dhclient[1074]: DHCPREQUEST on ens2 to 255.255.255.255 port 67 (xid=0x8baf276c) Aug 15 05:52:08 oleg133-client dhclient[1074]: DHCPACK from 192.168.201.254 (xid=0x8baf276c) Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.8903] dhcp4 (ens2): address 192.168.201.33 Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.8903] dhcp4 (ens2): plen 24 (255.255.255.0) Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.8904] dhcp4 (ens2): gateway 192.168.201.254 Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.8904] dhcp4 (ens2): lease time 60000 Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.8904] dhcp4 (ens2): hostname 'oleg133-client.virtnet' Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.8904] dhcp4 (ens2): nameserver '192.168.200.253' Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.8905] dhcp4 (ens2): domain name 'virtnet' Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.8905] dhcp4 (ens2): state changed new lease, address=192.168.201.33 Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.8919] policy: set 'ens2' (ens2) as default for IPv4 routing and DNS Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.8995] device (ens2): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'assume') Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.9070] device (ens2): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'assume') Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.9077] device (ens2): state change: secondaries -> activated (reason 'none', sys-iface-state: 'assume') Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.9087] manager: NetworkManager state is now CONNECTED_SITE Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.9111] device (ens2): Activation: successful, device activated. Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.9135] manager: NetworkManager state is now CONNECTED_GLOBAL Aug 15 05:52:08 oleg133-client NetworkManager[1032]: [1786787528.9165] manager: startup complete Aug 15 05:52:08 oleg133-client systemd[1]: Started Network Manager Wait Online. Aug 15 05:52:08 oleg133-client systemd[1]: Reached target Network is Online. Aug 15 05:52:08 oleg133-client systemd[1]: Starting Notify NFS peers of a restart... Aug 15 05:52:08 oleg133-client dhclient[1074]: bound to 192.168.201.33 -- renewal in 28214 seconds. Aug 15 05:52:08 oleg133-client systemd[1]: Starting Crash recovery kernel arming... Aug 15 05:52:08 oleg133-client systemd[1]: Starting System Logging Service... Aug 15 05:52:08 oleg133-client sm-notify[1093]: Version 2.3.3 starting Aug 15 05:52:08 oleg133-client systemd[1]: Started Notify NFS peers of a restart. Aug 15 05:52:09 oleg133-client systemd[1]: Started System Logging Service. Aug 15 05:52:09 oleg133-client rsyslogd[1095]: [origin software="rsyslogd" swVersion="8.2102.0-15.el8_10.1" x-pid="1095" x-info="https://www.rsyslog.com"] start Aug 15 05:52:09 oleg133-client rsyslogd[1095]: imjournal: No statefile exists, /var/lib/rsyslog/imjournal.state will be created (ignore if this is first run): No such file or directory [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/2040 ] Aug 15 05:52:09 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 05:52:10 oleg133-client kdumpctl[1168]: basename: missing operand Aug 15 05:52:10 oleg133-client kdumpctl[1168]: Try 'basename --help' for more information. Aug 15 05:52:11 oleg133-client dbus-daemon[1031]: [system] Activating via systemd: service name='org.freedesktop.PolicyKit1' unit='polkit.service' requested by ':1.7' (uid=0 pid=1053 comm="/usr/libexec/platform-python -Es /usr/sbin/tuned -") Aug 15 05:52:11 oleg133-client systemd[1]: Starting Authorization Manager... Aug 15 05:52:11 oleg133-client polkitd[1381]: Started polkitd version 0.115 Aug 15 05:52:12 oleg133-client polkitd[1381]: Loading rules from directory /etc/polkit-1/rules.d Aug 15 05:52:12 oleg133-client polkitd[1381]: Loading rules from directory /usr/share/polkit-1/rules.d Aug 15 05:52:12 oleg133-client polkitd[1381]: Finished loading, compiling and executing 2 rules Aug 15 05:52:12 oleg133-client dbus-daemon[1031]: [system] Successfully activated service 'org.freedesktop.PolicyKit1' Aug 15 05:52:12 oleg133-client systemd[1]: Started Authorization Manager. Aug 15 05:52:12 oleg133-client polkitd[1381]: Acquired the name org.freedesktop.PolicyKit1 on the system bus Aug 15 05:52:12 oleg133-client systemd[1]: Started Dynamic System Tuning Daemon. Aug 15 05:52:12 oleg133-client systemd[1]: Reached target Multi-User System. Aug 15 05:52:12 oleg133-client systemd[1]: Reached target Graphical Interface. Aug 15 05:52:12 oleg133-client systemd[1]: Starting Update UTMP about System Runlevel Changes... Aug 15 05:52:12 oleg133-client systemd[1]: systemd-update-utmp-runlevel.service: Succeeded. Aug 15 05:52:12 oleg133-client systemd[1]: Started Update UTMP about System Runlevel Changes. Aug 15 05:52:13 oleg133-client rngd[1030]: [jitter]: Unable to obtain AES key, disabling JITTER source Aug 15 05:52:13 oleg133-client rngd[1030]: [jitter]: Initialization Failed Aug 15 05:52:13 oleg133-client rngd[1030]: Process privileges have been dropped to 2:2 Aug 15 05:52:15 oleg133-client kdumpctl[1100]: kdump: kexec: loaded kdump kernel Aug 15 05:52:15 oleg133-client kdumpctl[1100]: kdump: Starting kdump: [OK] Aug 15 05:52:15 oleg133-client systemd[1]: Started Crash recovery kernel arming. Aug 15 05:52:15 oleg133-client systemd[1]: Startup finished in 3.054s (kernel) + 8.220s (initrd) + 11.801s (userspace) = 23.077s. Aug 15 05:52:17 oleg133-client sshd[1474]: Accepted publickey for root from 192.168.200.253 port 46544 ssh2: RSA SHA256:9e8LCkrakioBi3TKSwi0ueMHJ3ih9q2KxjmcAXV6W0I Aug 15 05:52:17 oleg133-client systemd[1]: Created slice User Slice of UID 0. Aug 15 05:52:17 oleg133-client systemd[1]: Starting User runtime directory /run/user/0... Aug 15 05:52:17 oleg133-client systemd-logind[1034]: New session 1 of user root. Aug 15 05:52:18 oleg133-client systemd[1]: Started User runtime directory /run/user/0. Aug 15 05:52:18 oleg133-client systemd[1]: Starting User Manager for UID 0... Aug 15 05:52:18 oleg133-client systemd[1489]: pam_unix(systemd-user:session): session opened for user root by (uid=0) Aug 15 05:52:18 oleg133-client systemd[1489]: Reached target Paths. Aug 15 05:52:18 oleg133-client systemd[1489]: Starting D-Bus User Message Bus Socket. Aug 15 05:52:18 oleg133-client systemd[1489]: Reached target Timers. Aug 15 05:52:18 oleg133-client systemd[1489]: Listening on D-Bus User Message Bus Socket. Aug 15 05:52:18 oleg133-client systemd[1489]: Reached target Sockets. Aug 15 05:52:18 oleg133-client systemd[1489]: Reached target Basic System. Aug 15 05:52:18 oleg133-client systemd[1489]: Reached target Default. Aug 15 05:52:18 oleg133-client systemd[1489]: Startup finished in 156ms. Aug 15 05:52:18 oleg133-client systemd[1]: Started User Manager for UID 0. Aug 15 05:52:18 oleg133-client systemd[1]: Started Session 1 of user root. Aug 15 05:52:18 oleg133-client sshd[1474]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:52:18 oleg133-client sshd[1500]: Received disconnect from 192.168.200.253 port 46544:11: disconnected by user Aug 15 05:52:18 oleg133-client sshd[1500]: Disconnected from user root 192.168.200.253 port 46544 Aug 15 05:52:18 oleg133-client sshd[1474]: pam_unix(sshd:session): session closed for user root Aug 15 05:52:18 oleg133-client systemd-logind[1034]: Session 1 logged out. Waiting for processes to exit. Aug 15 05:52:18 oleg133-client systemd[1]: session-1.scope: Succeeded. Aug 15 05:52:18 oleg133-client systemd-logind[1034]: Removed session 1. Aug 15 05:52:18 oleg133-client sshd[1536]: Accepted publickey for root from 192.168.200.253 port 46558 ssh2: RSA SHA256:9e8LCkrakioBi3TKSwi0ueMHJ3ih9q2KxjmcAXV6W0I Aug 15 05:52:18 oleg133-client systemd-logind[1034]: New session 3 of user root. Aug 15 05:52:18 oleg133-client systemd[1]: Started Session 3 of user root. Aug 15 05:52:18 oleg133-client sshd[1536]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:52:19 oleg133-client systemd[1]: NetworkManager-dispatcher.service: Succeeded. Aug 15 05:52:38 oleg133-client systemd[1]: systemd-hostnamed.service: Succeeded. Aug 15 05:52:59 oleg133-client systemd-logind[1034]: Watching system buttons on /dev/input/event0 (Power Button) Aug 15 05:52:59 oleg133-client systemd-udevd[2252]: Using default interface naming scheme 'rhel-8.0'. Aug 15 05:52:59 oleg133-client kernel: libcfs: loading out-of-tree module taints kernel. Aug 15 05:52:59 oleg133-client systemd-logind[1034]: Watching system buttons on /dev/input/event1 (AT Translated Set 2 keyboard) Aug 15 05:52:59 oleg133-client kernel: Key type ._llcrypt registered Aug 15 05:52:59 oleg133-client kernel: Key type .llcrypt registered Aug 15 05:53:00 oleg133-client kernel: libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 Aug 15 05:53:00 oleg133-client kernel: alg: No test for adler32 (adler32-zlib) Aug 15 05:53:01 oleg133-client kernel: Lustre: Lustre: Build Version: 2.17.57_1_g05abe65 Aug 15 05:53:02 oleg133-client kernel: LNet: Added LNI 192.168.201.33@tcp [8/256/0/180] Aug 15 05:53:04 oleg133-client kernel: Key type lgssc registered Aug 15 05:53:06 oleg133-client kernel: Lustre: Echo OBD driver; http://www.lustre.org/ Aug 15 05:54:09 oleg133-client systemd-logind[1034]: Watching system buttons on /dev/input/event0 (Power Button) Aug 15 05:54:09 oleg133-client systemd-udevd[4209]: Using default interface naming scheme 'rhel-8.0'. Aug 15 05:54:09 oleg133-client systemd-logind[1034]: Watching system buttons on /dev/input/event1 (AT Translated Set 2 keyboard) Aug 15 05:55:18 oleg133-client kernel: hrtimer: interrupt took 3153896 ns Aug 15 05:55:33 oleg133-client kernel: Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 Aug 15 05:55:35 oleg133-client sshd[5674]: Accepted publickey for root from 192.168.201.33 port 38166 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:55:35 oleg133-client systemd-logind[1034]: New session 4 of user root. Aug 15 05:55:35 oleg133-client systemd[1]: Started Session 4 of user root. Aug 15 05:55:35 oleg133-client sshd[5674]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:55:35 oleg133-client sshd[5687]: Received disconnect from 192.168.201.33 port 38166:11: disconnected by user Aug 15 05:55:35 oleg133-client sshd[5687]: Disconnected from user root 192.168.201.33 port 38166 Aug 15 05:55:35 oleg133-client sshd[5674]: pam_unix(sshd:session): session closed for user root Aug 15 05:55:35 oleg133-client systemd[1]: session-4.scope: Succeeded. Aug 15 05:55:35 oleg133-client systemd-logind[1034]: Session 4 logged out. Waiting for processes to exit. Aug 15 05:55:35 oleg133-client systemd-logind[1034]: Removed session 4. Aug 15 05:55:38 oleg133-client sshd[5847]: Accepted publickey for root from 192.168.201.33 port 55888 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:55:38 oleg133-client systemd-logind[1034]: New session 5 of user root. Aug 15 05:55:38 oleg133-client systemd[1]: Started Session 5 of user root. Aug 15 05:55:38 oleg133-client sshd[5847]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:55:38 oleg133-client kernel: Lustre: DEBUG MARKER: Using TIMEOUT=20 Aug 15 05:55:38 oleg133-client sshd[5860]: Received disconnect from 192.168.201.33 port 55888:11: disconnected by user Aug 15 05:55:38 oleg133-client sshd[5860]: Disconnected from user root 192.168.201.33 port 55888 Aug 15 05:55:38 oleg133-client sshd[5847]: pam_unix(sshd:session): session closed for user root Aug 15 05:55:38 oleg133-client systemd[1]: session-5.scope: Succeeded. Aug 15 05:55:38 oleg133-client systemd-logind[1034]: Session 5 logged out. Waiting for processes to exit. Aug 15 05:55:38 oleg133-client systemd-logind[1034]: Removed session 5. Aug 15 05:55:39 oleg133-client sshd[5980]: Accepted publickey for root from 192.168.201.33 port 55892 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:55:39 oleg133-client systemd-logind[1034]: New session 6 of user root. Aug 15 05:55:39 oleg133-client systemd[1]: Started Session 6 of user root. Aug 15 05:55:39 oleg133-client sshd[5980]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:55:40 oleg133-client sshd[5993]: Received disconnect from 192.168.201.33 port 55892:11: disconnected by user Aug 15 05:55:40 oleg133-client sshd[5993]: Disconnected from user root 192.168.201.33 port 55892 Aug 15 05:55:40 oleg133-client sshd[5980]: pam_unix(sshd:session): session closed for user root Aug 15 05:55:40 oleg133-client systemd[1]: session-6.scope: Succeeded. Aug 15 05:55:40 oleg133-client systemd-logind[1034]: Session 6 logged out. Waiting for processes to exit. Aug 15 05:55:40 oleg133-client systemd-logind[1034]: Removed session 6. Aug 15 05:55:47 oleg133-client systemd-udevd[6147]: Process '/home/green/git/lustre-release/lustre/utils/lctl set_param 'lod.*.mdt_hash=crush'' failed with exit code 2. Aug 15 05:55:50 oleg133-client sshd[6268]: Accepted publickey for root from 192.168.201.33 port 54650 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:55:50 oleg133-client systemd[1]: Started Session 7 of user root. Aug 15 05:55:50 oleg133-client systemd-logind[1034]: New session 7 of user root. Aug 15 05:55:50 oleg133-client sshd[6268]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:55:52 oleg133-client kernel: Lustre: DEBUG MARKER: oleg133-client.virtnet: executing check_logdir /tmp/testlogs/ Aug 15 05:55:52 oleg133-client sshd[6281]: Received disconnect from 192.168.201.33 port 54650:11: disconnected by user Aug 15 05:55:52 oleg133-client sshd[6281]: Disconnected from user root 192.168.201.33 port 54650 Aug 15 05:55:52 oleg133-client sshd[6268]: pam_unix(sshd:session): session closed for user root Aug 15 05:55:52 oleg133-client systemd[1]: session-7.scope: Succeeded. Aug 15 05:55:52 oleg133-client systemd-logind[1034]: Session 7 logged out. Waiting for processes to exit. Aug 15 05:55:52 oleg133-client systemd-logind[1034]: Removed session 7. Aug 15 05:55:56 oleg133-client sshd[6548]: Accepted publickey for root from 192.168.201.33 port 52610 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:55:56 oleg133-client systemd[1]: Started Session 8 of user root. Aug 15 05:55:56 oleg133-client systemd-logind[1034]: New session 8 of user root. Aug 15 05:55:56 oleg133-client sshd[6548]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:55:59 oleg133-client kernel: Lustre: lustre-OST0000-osc-ffff993747ebd000: disconnect after 24s idle Aug 15 05:55:59 oleg133-client kernel: Lustre: DEBUG MARKER: oleg133-client.virtnet: executing yml_node Aug 15 05:56:00 oleg133-client sshd[6561]: Received disconnect from 192.168.201.33 port 52610:11: disconnected by user Aug 15 05:56:00 oleg133-client sshd[6561]: Disconnected from user root 192.168.201.33 port 52610 Aug 15 05:56:00 oleg133-client sshd[6548]: pam_unix(sshd:session): session closed for user root Aug 15 05:56:00 oleg133-client systemd[1]: session-8.scope: Succeeded. Aug 15 05:56:00 oleg133-client systemd-logind[1034]: Session 8 logged out. Waiting for processes to exit. Aug 15 05:56:00 oleg133-client systemd-logind[1034]: Removed session 8. Aug 15 05:56:05 oleg133-client sshd[7143]: Accepted publickey for root from 192.168.201.33 port 52614 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:56:05 oleg133-client systemd-logind[1034]: New session 9 of user root. Aug 15 05:56:05 oleg133-client systemd[1]: Started Session 9 of user root. Aug 15 05:56:05 oleg133-client sshd[7143]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:56:06 oleg133-client kernel: Lustre: DEBUG MARKER: Client: 2.17.57.1 Aug 15 05:56:06 oleg133-client sshd[7156]: Received disconnect from 192.168.201.33 port 52614:11: disconnected by user Aug 15 05:56:06 oleg133-client sshd[7156]: Disconnected from user root 192.168.201.33 port 52614 Aug 15 05:56:06 oleg133-client sshd[7143]: pam_unix(sshd:session): session closed for user root Aug 15 05:56:06 oleg133-client systemd[1]: session-9.scope: Succeeded. Aug 15 05:56:06 oleg133-client systemd-logind[1034]: Session 9 logged out. Waiting for processes to exit. Aug 15 05:56:07 oleg133-client systemd-logind[1034]: Removed session 9. Aug 15 05:56:09 oleg133-client sshd[7299]: Accepted publickey for root from 192.168.201.33 port 34724 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:56:09 oleg133-client systemd-logind[1034]: New session 10 of user root. Aug 15 05:56:09 oleg133-client systemd[1]: Started Session 10 of user root. Aug 15 05:56:09 oleg133-client sshd[7299]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:56:09 oleg133-client kernel: Lustre: DEBUG MARKER: MDS: 2.17.57.1 Aug 15 05:56:09 oleg133-client sshd[7312]: Received disconnect from 192.168.201.33 port 34724:11: disconnected by user Aug 15 05:56:09 oleg133-client sshd[7312]: Disconnected from user root 192.168.201.33 port 34724 Aug 15 05:56:09 oleg133-client sshd[7299]: pam_unix(sshd:session): session closed for user root Aug 15 05:56:09 oleg133-client systemd[1]: session-10.scope: Succeeded. Aug 15 05:56:09 oleg133-client systemd-logind[1034]: Session 10 logged out. Waiting for processes to exit. Aug 15 05:56:10 oleg133-client systemd-logind[1034]: Removed session 10. Aug 15 05:56:13 oleg133-client sshd[7453]: Accepted publickey for root from 192.168.201.33 port 34732 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:56:13 oleg133-client systemd[1]: Started Session 11 of user root. Aug 15 05:56:13 oleg133-client systemd-logind[1034]: New session 11 of user root. Aug 15 05:56:13 oleg133-client sshd[7453]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:56:13 oleg133-client kernel: Lustre: DEBUG MARKER: OSS: 2.17.57.1 Aug 15 05:56:13 oleg133-client sshd[7467]: Received disconnect from 192.168.201.33 port 34732:11: disconnected by user Aug 15 05:56:13 oleg133-client sshd[7467]: Disconnected from user root 192.168.201.33 port 34732 Aug 15 05:56:13 oleg133-client sshd[7453]: pam_unix(sshd:session): session closed for user root Aug 15 05:56:13 oleg133-client systemd[1]: session-11.scope: Succeeded. Aug 15 05:56:13 oleg133-client systemd-logind[1034]: Session 11 logged out. Waiting for processes to exit. Aug 15 05:56:13 oleg133-client systemd-logind[1034]: Removed session 11. Aug 15 05:56:15 oleg133-client sshd[7599]: Accepted publickey for root from 192.168.201.33 port 34740 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:56:15 oleg133-client systemd-logind[1034]: New session 12 of user root. Aug 15 05:56:15 oleg133-client systemd[1]: Started Session 12 of user root. Aug 15 05:56:15 oleg133-client sshd[7599]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:56:15 oleg133-client kernel: Lustre: DEBUG MARKER: -----============= acceptance-small: recovery-small ============----- Sat Aug 15 05:56:13 EDT 2026 Aug 15 05:56:15 oleg133-client sshd[7612]: Received disconnect from 192.168.201.33 port 34740:11: disconnected by user Aug 15 05:56:15 oleg133-client sshd[7612]: Disconnected from user root 192.168.201.33 port 34740 Aug 15 05:56:15 oleg133-client sshd[7599]: pam_unix(sshd:session): session closed for user root Aug 15 05:56:15 oleg133-client systemd[1]: session-12.scope: Succeeded. Aug 15 05:56:15 oleg133-client systemd-logind[1034]: Session 12 logged out. Waiting for processes to exit. Aug 15 05:56:15 oleg133-client systemd-logind[1034]: Removed session 12. Aug 15 05:56:34 oleg133-client sshd[8115]: Accepted publickey for root from 192.168.201.33 port 47400 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:56:34 oleg133-client systemd-logind[1034]: New session 13 of user root. Aug 15 05:56:34 oleg133-client systemd[1]: Started Session 13 of user root. Aug 15 05:56:34 oleg133-client sshd[8115]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:56:35 oleg133-client kernel: Lustre: DEBUG MARKER: excepting tests: 136 Aug 15 05:56:35 oleg133-client sshd[8128]: Received disconnect from 192.168.201.33 port 47400:11: disconnected by user Aug 15 05:56:35 oleg133-client sshd[8128]: Disconnected from user root 192.168.201.33 port 47400 Aug 15 05:56:35 oleg133-client sshd[8115]: pam_unix(sshd:session): session closed for user root Aug 15 05:56:35 oleg133-client systemd[1]: session-13.scope: Succeeded. Aug 15 05:56:35 oleg133-client systemd-logind[1034]: Session 13 logged out. Waiting for processes to exit. Aug 15 05:56:35 oleg133-client systemd-logind[1034]: Removed session 13. Aug 15 05:56:36 oleg133-client sshd[8297]: Accepted publickey for root from 192.168.201.33 port 46564 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:56:36 oleg133-client systemd[1]: Started Session 14 of user root. Aug 15 05:56:36 oleg133-client systemd-logind[1034]: New session 14 of user root. Aug 15 05:56:36 oleg133-client sshd[8297]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:56:37 oleg133-client kernel: Lustre: DEBUG MARKER: === recovery-small: start setup 05:56:35 (1786787795) === Aug 15 05:56:37 oleg133-client sshd[8310]: Received disconnect from 192.168.201.33 port 46564:11: disconnected by user Aug 15 05:56:37 oleg133-client sshd[8310]: Disconnected from user root 192.168.201.33 port 46564 Aug 15 05:56:37 oleg133-client sshd[8297]: pam_unix(sshd:session): session closed for user root Aug 15 05:56:37 oleg133-client systemd-logind[1034]: Session 14 logged out. Waiting for processes to exit. Aug 15 05:56:37 oleg133-client systemd[1]: session-14.scope: Succeeded. Aug 15 05:56:37 oleg133-client systemd-logind[1034]: Removed session 14. Aug 15 05:56:43 oleg133-client sshd[8565]: Accepted publickey for root from 192.168.201.33 port 46574 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:56:43 oleg133-client systemd-logind[1034]: New session 15 of user root. Aug 15 05:56:43 oleg133-client systemd[1]: Started Session 15 of user root. Aug 15 05:56:43 oleg133-client sshd[8565]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:56:43 oleg133-client kernel: Lustre: DEBUG MARKER: oleg133-client.virtnet: executing check_config_client /mnt/lustre Aug 15 05:56:43 oleg133-client sshd[8578]: Received disconnect from 192.168.201.33 port 46574:11: disconnected by user Aug 15 05:56:43 oleg133-client sshd[8578]: Disconnected from user root 192.168.201.33 port 46574 Aug 15 05:56:43 oleg133-client sshd[8565]: pam_unix(sshd:session): session closed for user root Aug 15 05:56:44 oleg133-client systemd[1]: session-15.scope: Succeeded. Aug 15 05:56:44 oleg133-client systemd-logind[1034]: Session 15 logged out. Waiting for processes to exit. Aug 15 05:56:44 oleg133-client systemd-logind[1034]: Removed session 15. Aug 15 05:56:59 oleg133-client sshd[9222]: Accepted publickey for root from 192.168.201.33 port 44076 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:57:00 oleg133-client systemd-logind[1034]: New session 16 of user root. Aug 15 05:57:00 oleg133-client systemd[1]: Started Session 16 of user root. Aug 15 05:57:00 oleg133-client sshd[9222]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:57:00 oleg133-client kernel: Lustre: DEBUG MARKER: Using TIMEOUT=20 Aug 15 05:57:00 oleg133-client sshd[9235]: Received disconnect from 192.168.201.33 port 44076:11: disconnected by user Aug 15 05:57:00 oleg133-client sshd[9235]: Disconnected from user root 192.168.201.33 port 44076 Aug 15 05:57:00 oleg133-client sshd[9222]: pam_unix(sshd:session): session closed for user root Aug 15 05:57:00 oleg133-client systemd[1]: session-16.scope: Succeeded. Aug 15 05:57:00 oleg133-client systemd-logind[1034]: Session 16 logged out. Waiting for processes to exit. Aug 15 05:57:00 oleg133-client systemd-logind[1034]: Removed session 16. Aug 15 05:57:02 oleg133-client sshd[9354]: Accepted publickey for root from 192.168.201.33 port 44084 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:57:02 oleg133-client systemd-logind[1034]: New session 17 of user root. Aug 15 05:57:02 oleg133-client systemd[1]: Started Session 17 of user root. Aug 15 05:57:02 oleg133-client sshd[9354]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:57:02 oleg133-client sshd[9367]: Received disconnect from 192.168.201.33 port 44084:11: disconnected by user Aug 15 05:57:02 oleg133-client sshd[9367]: Disconnected from user root 192.168.201.33 port 44084 Aug 15 05:57:02 oleg133-client sshd[9354]: pam_unix(sshd:session): session closed for user root Aug 15 05:57:02 oleg133-client systemd[1]: session-17.scope: Succeeded. Aug 15 05:57:02 oleg133-client systemd-logind[1034]: Session 17 logged out. Waiting for processes to exit. Aug 15 05:57:02 oleg133-client systemd-logind[1034]: Removed session 17. Aug 15 05:57:12 oleg133-client sshd[9687]: Accepted publickey for root from 192.168.201.33 port 59830 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:57:13 oleg133-client systemd[1]: Started Session 18 of user root. Aug 15 05:57:13 oleg133-client systemd-logind[1034]: New session 18 of user root. Aug 15 05:57:13 oleg133-client sshd[9687]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:57:13 oleg133-client sshd[9700]: Received disconnect from 192.168.201.33 port 59830:11: disconnected by user Aug 15 05:57:13 oleg133-client sshd[9700]: Disconnected from user root 192.168.201.33 port 59830 Aug 15 05:57:13 oleg133-client sshd[9687]: pam_unix(sshd:session): session closed for user root Aug 15 05:57:13 oleg133-client systemd-logind[1034]: Session 18 logged out. Waiting for processes to exit. Aug 15 05:57:13 oleg133-client systemd[1]: session-18.scope: Succeeded. Aug 15 05:57:13 oleg133-client systemd-logind[1034]: Removed session 18. Aug 15 05:57:14 oleg133-client sshd[9824]: Accepted publickey for root from 192.168.201.33 port 59842 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:57:14 oleg133-client systemd-logind[1034]: New session 19 of user root. Aug 15 05:57:14 oleg133-client systemd[1]: Started Session 19 of user root. Aug 15 05:57:14 oleg133-client sshd[9824]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:57:15 oleg133-client kernel: Lustre: DEBUG MARKER: === recovery-small: finish setup 05:57:13 (1786787833) === Aug 15 05:57:15 oleg133-client sshd[9837]: Received disconnect from 192.168.201.33 port 59842:11: disconnected by user Aug 15 05:57:15 oleg133-client sshd[9837]: Disconnected from user root 192.168.201.33 port 59842 Aug 15 05:57:15 oleg133-client sshd[9824]: pam_unix(sshd:session): session closed for user root Aug 15 05:57:15 oleg133-client systemd[1]: session-19.scope: Succeeded. Aug 15 05:57:15 oleg133-client systemd-logind[1034]: Session 19 logged out. Waiting for processes to exit. Aug 15 05:57:15 oleg133-client systemd-logind[1034]: Removed session 19. Aug 15 05:57:17 oleg133-client sshd[9977]: Accepted publickey for root from 192.168.201.33 port 57808 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:57:17 oleg133-client systemd-logind[1034]: New session 20 of user root. Aug 15 05:57:17 oleg133-client systemd[1]: Started Session 20 of user root. Aug 15 05:57:17 oleg133-client sshd[9977]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:57:17 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 1: create, chmod, stat: drop req, drop rep ========================================================== 05:57:15 (1786787835) Aug 15 05:57:17 oleg133-client sshd[9990]: Received disconnect from 192.168.201.33 port 57808:11: disconnected by user Aug 15 05:57:17 oleg133-client sshd[9990]: Disconnected from user root 192.168.201.33 port 57808 Aug 15 05:57:17 oleg133-client sshd[9977]: pam_unix(sshd:session): session closed for user root Aug 15 05:57:18 oleg133-client systemd[1]: session-20.scope: Succeeded. Aug 15 05:57:18 oleg133-client systemd-logind[1034]: Session 20 logged out. Waiting for processes to exit. Aug 15 05:57:18 oleg133-client systemd-logind[1034]: Removed session 20. Aug 15 05:57:35 oleg133-client kernel: Lustre: 10039:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787838/real 1786787838] req@ffff993745396a00 x1873582575595904/t0(0) o700->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:30/10 lens 264/248 e 0 to 1 dl 1786787854 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'mcreate.0' uid:0 gid:0 projid:4294967295 Aug 15 05:57:35 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 05:57:35 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 05:57:53 oleg133-client kernel: Lustre: 10060:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787857/real 1786787857] req@ffff993760341f80 x1873582575597696/t0(0) o36->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 520/576 e 0 to 1 dl 1786787873 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'mcreate.0' uid:0 gid:0 projid:4294967295 Aug 15 05:57:53 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 05:57:54 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 05:58:11 oleg133-client kernel: Lustre: 10086:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787875/real 1786787875] req@ffff993760046680 x1873582575598976/t0(0) o101->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 576/1152 e 0 to 1 dl 1786787891 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'tchmod.0' uid:0 gid:0 projid:0 Aug 15 05:58:11 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 05:58:11 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 05:58:29 oleg133-client kernel: Lustre: 10107:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787893/real 1786787893] req@ffff9937600a9c00 x1873582575601152/t0(0) o36->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 488/512 e 0 to 1 dl 1786787909 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'tchmod.0' uid:0 gid:0 projid:4294967295 Aug 15 05:58:29 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 05:58:29 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 05:58:47 oleg133-client kernel: Lustre: 10133:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787911/real 1786787911] req@ffff993760046d80 x1873582575602432/t0(0) o34->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 472/728 e 0 to 1 dl 1786787927 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'statone.0' uid:0 gid:0 projid:0 Aug 15 05:58:47 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 05:58:47 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 05:59:05 oleg133-client kernel: Lustre: 10155:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787929/real 1786787929] req@ffff9937600a8700 x1873582575603712/t0(0) o34->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 472/728 e 0 to 1 dl 1786787945 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'statone.0' uid:0 gid:0 projid:0 Aug 15 05:59:05 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 05:59:05 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 05:59:09 oleg133-client sshd[10253]: Accepted publickey for root from 192.168.201.33 port 58946 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:59:09 oleg133-client systemd-logind[1034]: New session 21 of user root. Aug 15 05:59:09 oleg133-client systemd[1]: Started Session 21 of user root. Aug 15 05:59:09 oleg133-client sshd[10253]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:59:10 oleg133-client sshd[10266]: Received disconnect from 192.168.201.33 port 58946:11: disconnected by user Aug 15 05:59:10 oleg133-client sshd[10266]: Disconnected from user root 192.168.201.33 port 58946 Aug 15 05:59:10 oleg133-client sshd[10253]: pam_unix(sshd:session): session closed for user root Aug 15 05:59:10 oleg133-client systemd[1]: session-21.scope: Succeeded. Aug 15 05:59:10 oleg133-client systemd-logind[1034]: Session 21 logged out. Waiting for processes to exit. Aug 15 05:59:10 oleg133-client systemd-logind[1034]: Removed session 21. Aug 15 05:59:11 oleg133-client sshd[10390]: Accepted publickey for root from 192.168.201.33 port 58954 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:59:11 oleg133-client systemd-logind[1034]: New session 22 of user root. Aug 15 05:59:11 oleg133-client systemd[1]: Started Session 22 of user root. Aug 15 05:59:11 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 05:59:11 oleg133-client sshd[10390]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:59:12 oleg133-client sshd[10404]: Received disconnect from 192.168.201.33 port 58954:11: disconnected by user Aug 15 05:59:12 oleg133-client sshd[10404]: Disconnected from user root 192.168.201.33 port 58954 Aug 15 05:59:12 oleg133-client sshd[10390]: pam_unix(sshd:session): session closed for user root Aug 15 05:59:12 oleg133-client systemd[1]: session-22.scope: Succeeded. Aug 15 05:59:12 oleg133-client systemd-logind[1034]: Session 22 logged out. Waiting for processes to exit. Aug 15 05:59:12 oleg133-client systemd-logind[1034]: Removed session 22. Aug 15 05:59:13 oleg133-client sshd[10535]: Accepted publickey for root from 192.168.201.33 port 58968 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:59:13 oleg133-client systemd-logind[1034]: New session 23 of user root. Aug 15 05:59:13 oleg133-client systemd[1]: Started Session 23 of user root. Aug 15 05:59:13 oleg133-client sshd[10535]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:59:14 oleg133-client sshd[10548]: Received disconnect from 192.168.201.33 port 58968:11: disconnected by user Aug 15 05:59:14 oleg133-client sshd[10548]: Disconnected from user root 192.168.201.33 port 58968 Aug 15 05:59:14 oleg133-client sshd[10535]: pam_unix(sshd:session): session closed for user root Aug 15 05:59:14 oleg133-client systemd-logind[1034]: Session 23 logged out. Waiting for processes to exit. Aug 15 05:59:14 oleg133-client systemd[1]: session-23.scope: Succeeded. Aug 15 05:59:14 oleg133-client systemd-logind[1034]: Removed session 23. Aug 15 05:59:15 oleg133-client sshd[10690]: Accepted publickey for root from 192.168.201.33 port 49184 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:59:15 oleg133-client systemd-logind[1034]: New session 24 of user root. Aug 15 05:59:15 oleg133-client systemd[1]: Started Session 24 of user root. Aug 15 05:59:15 oleg133-client sshd[10690]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:59:16 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 4: open: drop req, drop rep ======= 05:59:14 (1786787954) Aug 15 05:59:16 oleg133-client sshd[10704]: Received disconnect from 192.168.201.33 port 49184:11: disconnected by user Aug 15 05:59:16 oleg133-client sshd[10704]: Disconnected from user root 192.168.201.33 port 49184 Aug 15 05:59:16 oleg133-client sshd[10690]: pam_unix(sshd:session): session closed for user root Aug 15 05:59:16 oleg133-client systemd[1]: session-24.scope: Succeeded. Aug 15 05:59:16 oleg133-client systemd-logind[1034]: Session 24 logged out. Waiting for processes to exit. Aug 15 05:59:16 oleg133-client systemd-logind[1034]: Removed session 24. Aug 15 05:59:33 oleg133-client kernel: Lustre: 10760:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787957/real 1786787957] req@ffff9937600a8700 x1873582575606272/t0(0) o101->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 576/1152 e 0 to 1 dl 1786787973 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 Aug 15 05:59:33 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 05:59:33 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 05:59:55 oleg133-client sshd[10878]: Accepted publickey for root from 192.168.201.33 port 43622 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:59:55 oleg133-client systemd-logind[1034]: New session 25 of user root. Aug 15 05:59:55 oleg133-client systemd[1]: Started Session 25 of user root. Aug 15 05:59:55 oleg133-client sshd[10878]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:59:55 oleg133-client sshd[10891]: Received disconnect from 192.168.201.33 port 43622:11: disconnected by user Aug 15 05:59:55 oleg133-client sshd[10891]: Disconnected from user root 192.168.201.33 port 43622 Aug 15 05:59:55 oleg133-client sshd[10878]: pam_unix(sshd:session): session closed for user root Aug 15 05:59:55 oleg133-client systemd[1]: session-25.scope: Succeeded. Aug 15 05:59:55 oleg133-client systemd-logind[1034]: Session 25 logged out. Waiting for processes to exit. Aug 15 05:59:55 oleg133-client systemd-logind[1034]: Removed session 25. Aug 15 05:59:56 oleg133-client sshd[11016]: Accepted publickey for root from 192.168.201.33 port 58600 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:59:56 oleg133-client systemd-logind[1034]: New session 26 of user root. Aug 15 05:59:56 oleg133-client systemd[1]: Started Session 26 of user root. Aug 15 05:59:56 oleg133-client sshd[11016]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:59:56 oleg133-client sshd[11029]: Received disconnect from 192.168.201.33 port 58600:11: disconnected by user Aug 15 05:59:56 oleg133-client sshd[11029]: Disconnected from user root 192.168.201.33 port 58600 Aug 15 05:59:56 oleg133-client sshd[11016]: pam_unix(sshd:session): session closed for user root Aug 15 05:59:57 oleg133-client systemd-logind[1034]: Session 26 logged out. Waiting for processes to exit. Aug 15 05:59:57 oleg133-client systemd[1]: session-26.scope: Succeeded. Aug 15 05:59:57 oleg133-client systemd-logind[1034]: Removed session 26. Aug 15 05:59:58 oleg133-client sshd[11160]: Accepted publickey for root from 192.168.201.33 port 58608 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:59:58 oleg133-client systemd[1]: Started Session 27 of user root. Aug 15 05:59:58 oleg133-client systemd-logind[1034]: New session 27 of user root. Aug 15 05:59:58 oleg133-client sshd[11160]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 05:59:58 oleg133-client sshd[11173]: Received disconnect from 192.168.201.33 port 58608:11: disconnected by user Aug 15 05:59:58 oleg133-client sshd[11173]: Disconnected from user root 192.168.201.33 port 58608 Aug 15 05:59:58 oleg133-client sshd[11160]: pam_unix(sshd:session): session closed for user root Aug 15 05:59:58 oleg133-client systemd[1]: session-27.scope: Succeeded. Aug 15 05:59:58 oleg133-client systemd-logind[1034]: Session 27 logged out. Waiting for processes to exit. Aug 15 05:59:58 oleg133-client systemd-logind[1034]: Removed session 27. Aug 15 05:59:59 oleg133-client sshd[11313]: Accepted publickey for root from 192.168.201.33 port 58618 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 05:59:59 oleg133-client systemd-logind[1034]: New session 28 of user root. Aug 15 05:59:59 oleg133-client systemd[1]: Started Session 28 of user root. Aug 15 05:59:59 oleg133-client sshd[11313]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:00:00 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 5: rename: drop req, drop rep ===== 05:59:58 (1786787998) Aug 15 06:00:00 oleg133-client sshd[11326]: Received disconnect from 192.168.201.33 port 58618:11: disconnected by user Aug 15 06:00:00 oleg133-client sshd[11326]: Disconnected from user root 192.168.201.33 port 58618 Aug 15 06:00:00 oleg133-client sshd[11313]: pam_unix(sshd:session): session closed for user root Aug 15 06:00:00 oleg133-client systemd[1]: session-28.scope: Succeeded. Aug 15 06:00:00 oleg133-client systemd-logind[1034]: Session 28 logged out. Waiting for processes to exit. Aug 15 06:00:00 oleg133-client systemd-logind[1034]: Removed session 28. Aug 15 06:00:17 oleg133-client kernel: Lustre: 11382:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786788001/real 1786788001] req@ffff993760044000 x1873582575612416/t0(0) o101->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 576/1152 e 0 to 1 dl 1786788017 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'mv.0' uid:0 gid:0 projid:0 Aug 15 06:00:17 oleg133-client kernel: Lustre: 11382:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message Aug 15 06:00:17 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 06:00:17 oleg133-client kernel: Lustre: Skipped 1 previous similar message Aug 15 06:00:17 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 06:00:17 oleg133-client kernel: Lustre: Skipped 1 previous similar message Aug 15 06:00:38 oleg133-client sshd[11512]: Accepted publickey for root from 192.168.201.33 port 36112 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:00:38 oleg133-client systemd[1]: Started Session 29 of user root. Aug 15 06:00:38 oleg133-client systemd-logind[1034]: New session 29 of user root. Aug 15 06:00:38 oleg133-client sshd[11512]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:00:39 oleg133-client sshd[11525]: Received disconnect from 192.168.201.33 port 36112:11: disconnected by user Aug 15 06:00:39 oleg133-client sshd[11525]: Disconnected from user root 192.168.201.33 port 36112 Aug 15 06:00:39 oleg133-client sshd[11512]: pam_unix(sshd:session): session closed for user root Aug 15 06:00:39 oleg133-client systemd[1]: session-29.scope: Succeeded. Aug 15 06:00:39 oleg133-client systemd-logind[1034]: Session 29 logged out. Waiting for processes to exit. Aug 15 06:00:39 oleg133-client systemd-logind[1034]: Removed session 29. Aug 15 06:00:40 oleg133-client sshd[11651]: Accepted publickey for root from 192.168.201.33 port 36116 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:00:40 oleg133-client systemd[1]: Started Session 30 of user root. Aug 15 06:00:40 oleg133-client systemd-logind[1034]: New session 30 of user root. Aug 15 06:00:40 oleg133-client sshd[11651]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:00:40 oleg133-client sshd[11664]: Received disconnect from 192.168.201.33 port 36116:11: disconnected by user Aug 15 06:00:40 oleg133-client sshd[11664]: Disconnected from user root 192.168.201.33 port 36116 Aug 15 06:00:40 oleg133-client sshd[11651]: pam_unix(sshd:session): session closed for user root Aug 15 06:00:41 oleg133-client systemd-logind[1034]: Session 30 logged out. Waiting for processes to exit. Aug 15 06:00:41 oleg133-client systemd[1]: session-30.scope: Succeeded. Aug 15 06:00:41 oleg133-client systemd-logind[1034]: Removed session 30. Aug 15 06:00:41 oleg133-client sshd[11795]: Accepted publickey for root from 192.168.201.33 port 36124 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:00:41 oleg133-client systemd-logind[1034]: New session 31 of user root. Aug 15 06:00:41 oleg133-client systemd[1]: Started Session 31 of user root. Aug 15 06:00:41 oleg133-client sshd[11795]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:00:42 oleg133-client sshd[11808]: Received disconnect from 192.168.201.33 port 36124:11: disconnected by user Aug 15 06:00:42 oleg133-client sshd[11808]: Disconnected from user root 192.168.201.33 port 36124 Aug 15 06:00:42 oleg133-client sshd[11795]: pam_unix(sshd:session): session closed for user root Aug 15 06:00:42 oleg133-client systemd-logind[1034]: Session 31 logged out. Waiting for processes to exit. Aug 15 06:00:42 oleg133-client systemd[1]: session-31.scope: Succeeded. Aug 15 06:00:42 oleg133-client systemd-logind[1034]: Removed session 31. Aug 15 06:00:43 oleg133-client sshd[11948]: Accepted publickey for root from 192.168.201.33 port 36136 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:00:43 oleg133-client systemd-logind[1034]: New session 32 of user root. Aug 15 06:00:43 oleg133-client systemd[1]: Started Session 32 of user root. Aug 15 06:00:43 oleg133-client sshd[11948]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:00:44 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 6: link, unlink: drop req, drop rep ========================================================== 06:00:42 (1786788042) Aug 15 06:00:44 oleg133-client sshd[11961]: Received disconnect from 192.168.201.33 port 36136:11: disconnected by user Aug 15 06:00:44 oleg133-client sshd[11961]: Disconnected from user root 192.168.201.33 port 36136 Aug 15 06:00:44 oleg133-client sshd[11948]: pam_unix(sshd:session): session closed for user root Aug 15 06:00:44 oleg133-client systemd[1]: session-32.scope: Succeeded. Aug 15 06:00:44 oleg133-client systemd-logind[1034]: Session 32 logged out. Waiting for processes to exit. Aug 15 06:00:44 oleg133-client systemd-logind[1034]: Removed session 32. Aug 15 06:01:01 oleg133-client CROND[12021]: (root) CMD (run-parts /etc/cron.hourly) Aug 15 06:01:01 oleg133-client run-parts[12021]: (/etc/cron.hourly) starting 0anacron Aug 15 06:01:01 oleg133-client anacron[12030]: Anacron started on 2026-08-15 Aug 15 06:01:01 oleg133-client anacron[12030]: Will run job `cron.daily' in 43 min. Aug 15 06:01:01 oleg133-client anacron[12030]: Will run job `cron.weekly' in 63 min. Aug 15 06:01:01 oleg133-client anacron[12030]: Will run job `cron.monthly' in 83 min. Aug 15 06:01:01 oleg133-client anacron[12030]: Jobs will be executed sequentially Aug 15 06:01:01 oleg133-client run-parts[12021]: (/etc/cron.hourly) finished 0anacron Aug 15 06:01:01 oleg133-client kernel: Lustre: lustre-OST0001-osc-ffff993747ebd000: disconnect after 21s idle Aug 15 06:01:01 oleg133-client kernel: Lustre: Skipped 1 previous similar message Aug 15 06:01:41 oleg133-client kernel: Lustre: 12094:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786788085/real 1786788085] req@ffff993760047480 x1873582575625856/t0(0) o36->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 504/624 e 0 to 1 dl 1786788101 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'unlink.0' uid:0 gid:0 projid:4294967295 Aug 15 06:01:41 oleg133-client kernel: Lustre: 12094:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Aug 15 06:01:41 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 06:01:41 oleg133-client kernel: Lustre: Skipped 3 previous similar messages Aug 15 06:01:41 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 06:01:41 oleg133-client kernel: Lustre: Skipped 3 previous similar messages Aug 15 06:01:45 oleg133-client sshd[12197]: Accepted publickey for root from 192.168.201.33 port 55328 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:01:46 oleg133-client systemd-logind[1034]: New session 33 of user root. Aug 15 06:01:46 oleg133-client systemd[1]: Started Session 33 of user root. Aug 15 06:01:46 oleg133-client sshd[12197]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:01:46 oleg133-client sshd[12210]: Received disconnect from 192.168.201.33 port 55328:11: disconnected by user Aug 15 06:01:46 oleg133-client sshd[12210]: Disconnected from user root 192.168.201.33 port 55328 Aug 15 06:01:46 oleg133-client sshd[12197]: pam_unix(sshd:session): session closed for user root Aug 15 06:01:46 oleg133-client systemd[1]: session-33.scope: Succeeded. Aug 15 06:01:46 oleg133-client systemd-logind[1034]: Session 33 logged out. Waiting for processes to exit. Aug 15 06:01:46 oleg133-client systemd-logind[1034]: Removed session 33. Aug 15 06:01:47 oleg133-client sshd[12335]: Accepted publickey for root from 192.168.201.33 port 55334 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:01:47 oleg133-client systemd-logind[1034]: New session 34 of user root. Aug 15 06:01:47 oleg133-client systemd[1]: Started Session 34 of user root. Aug 15 06:01:47 oleg133-client sshd[12335]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:01:48 oleg133-client sshd[12348]: Received disconnect from 192.168.201.33 port 55334:11: disconnected by user Aug 15 06:01:48 oleg133-client sshd[12348]: Disconnected from user root 192.168.201.33 port 55334 Aug 15 06:01:48 oleg133-client sshd[12335]: pam_unix(sshd:session): session closed for user root Aug 15 06:01:48 oleg133-client systemd[1]: session-34.scope: Succeeded. Aug 15 06:01:48 oleg133-client systemd-logind[1034]: Session 34 logged out. Waiting for processes to exit. Aug 15 06:01:48 oleg133-client systemd-logind[1034]: Removed session 34. Aug 15 06:01:49 oleg133-client sshd[12482]: Accepted publickey for root from 192.168.201.33 port 55340 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:01:50 oleg133-client systemd-logind[1034]: New session 35 of user root. Aug 15 06:01:50 oleg133-client systemd[1]: Started Session 35 of user root. Aug 15 06:01:50 oleg133-client sshd[12482]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:01:50 oleg133-client sshd[12495]: Received disconnect from 192.168.201.33 port 55340:11: disconnected by user Aug 15 06:01:50 oleg133-client sshd[12495]: Disconnected from user root 192.168.201.33 port 55340 Aug 15 06:01:50 oleg133-client sshd[12482]: pam_unix(sshd:session): session closed for user root Aug 15 06:01:50 oleg133-client systemd-logind[1034]: Session 35 logged out. Waiting for processes to exit. Aug 15 06:01:50 oleg133-client systemd[1]: session-35.scope: Succeeded. Aug 15 06:01:50 oleg133-client systemd-logind[1034]: Removed session 35. Aug 15 06:01:52 oleg133-client sshd[12635]: Accepted publickey for root from 192.168.201.33 port 55352 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:01:52 oleg133-client systemd-logind[1034]: New session 36 of user root. Aug 15 06:01:52 oleg133-client systemd[1]: Started Session 36 of user root. Aug 15 06:01:52 oleg133-client sshd[12635]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:01:52 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 8: touch: drop rep (bug 1423) ===== 06:01:50 (1786788110) Aug 15 06:01:52 oleg133-client sshd[12648]: Received disconnect from 192.168.201.33 port 55352:11: disconnected by user Aug 15 06:01:52 oleg133-client sshd[12648]: Disconnected from user root 192.168.201.33 port 55352 Aug 15 06:01:52 oleg133-client sshd[12635]: pam_unix(sshd:session): session closed for user root Aug 15 06:01:53 oleg133-client systemd[1]: session-36.scope: Succeeded. Aug 15 06:01:53 oleg133-client systemd-logind[1034]: Session 36 logged out. Waiting for processes to exit. Aug 15 06:01:53 oleg133-client systemd-logind[1034]: Removed session 36. Aug 15 06:02:15 oleg133-client sshd[12789]: Accepted publickey for root from 192.168.201.33 port 56176 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:02:15 oleg133-client systemd-logind[1034]: New session 37 of user root. Aug 15 06:02:15 oleg133-client systemd[1]: Started Session 37 of user root. Aug 15 06:02:15 oleg133-client sshd[12789]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:02:16 oleg133-client sshd[12802]: Received disconnect from 192.168.201.33 port 56176:11: disconnected by user Aug 15 06:02:16 oleg133-client sshd[12802]: Disconnected from user root 192.168.201.33 port 56176 Aug 15 06:02:16 oleg133-client sshd[12789]: pam_unix(sshd:session): session closed for user root Aug 15 06:02:16 oleg133-client systemd-logind[1034]: Session 37 logged out. Waiting for processes to exit. Aug 15 06:02:16 oleg133-client systemd[1]: session-37.scope: Succeeded. Aug 15 06:02:16 oleg133-client systemd-logind[1034]: Removed session 37. Aug 15 06:02:17 oleg133-client sshd[12926]: Accepted publickey for root from 192.168.201.33 port 56648 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:02:18 oleg133-client systemd-logind[1034]: New session 38 of user root. Aug 15 06:02:18 oleg133-client systemd[1]: Started Session 38 of user root. Aug 15 06:02:18 oleg133-client sshd[12926]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:02:18 oleg133-client sshd[12939]: Received disconnect from 192.168.201.33 port 56648:11: disconnected by user Aug 15 06:02:18 oleg133-client sshd[12939]: Disconnected from user root 192.168.201.33 port 56648 Aug 15 06:02:18 oleg133-client sshd[12926]: pam_unix(sshd:session): session closed for user root Aug 15 06:02:18 oleg133-client systemd[1]: session-38.scope: Succeeded. Aug 15 06:02:18 oleg133-client systemd-logind[1034]: Session 38 logged out. Waiting for processes to exit. Aug 15 06:02:18 oleg133-client systemd-logind[1034]: Removed session 38. Aug 15 06:02:20 oleg133-client sshd[13071]: Accepted publickey for root from 192.168.201.33 port 56660 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:02:20 oleg133-client systemd-logind[1034]: New session 39 of user root. Aug 15 06:02:20 oleg133-client systemd[1]: Started Session 39 of user root. Aug 15 06:02:20 oleg133-client sshd[13071]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:02:20 oleg133-client sshd[13084]: Received disconnect from 192.168.201.33 port 56660:11: disconnected by user Aug 15 06:02:20 oleg133-client sshd[13084]: Disconnected from user root 192.168.201.33 port 56660 Aug 15 06:02:20 oleg133-client sshd[13071]: pam_unix(sshd:session): session closed for user root Aug 15 06:02:21 oleg133-client systemd[1]: session-39.scope: Succeeded. Aug 15 06:02:21 oleg133-client systemd-logind[1034]: Session 39 logged out. Waiting for processes to exit. Aug 15 06:02:21 oleg133-client systemd-logind[1034]: Removed session 39. Aug 15 06:02:22 oleg133-client sshd[13224]: Accepted publickey for root from 192.168.201.33 port 56666 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:02:22 oleg133-client systemd-logind[1034]: New session 40 of user root. Aug 15 06:02:22 oleg133-client systemd[1]: Started Session 40 of user root. Aug 15 06:02:22 oleg133-client sshd[13224]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:02:22 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 9: pause bulk on OST (bug 1420) === 06:02:21 (1786788141) Aug 15 06:02:22 oleg133-client sshd[13237]: Received disconnect from 192.168.201.33 port 56666:11: disconnected by user Aug 15 06:02:22 oleg133-client sshd[13237]: Disconnected from user root 192.168.201.33 port 56666 Aug 15 06:02:22 oleg133-client sshd[13224]: pam_unix(sshd:session): session closed for user root Aug 15 06:02:22 oleg133-client systemd-logind[1034]: Session 40 logged out. Waiting for processes to exit. Aug 15 06:02:23 oleg133-client systemd[1]: session-40.scope: Succeeded. Aug 15 06:02:23 oleg133-client systemd-logind[1034]: Removed session 40. Aug 15 06:02:34 oleg133-client sshd[13461]: Accepted publickey for root from 192.168.201.33 port 35932 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:02:34 oleg133-client systemd-logind[1034]: New session 41 of user root. Aug 15 06:02:34 oleg133-client systemd[1]: Started Session 41 of user root. Aug 15 06:02:34 oleg133-client sshd[13461]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:02:34 oleg133-client sshd[13474]: Received disconnect from 192.168.201.33 port 35932:11: disconnected by user Aug 15 06:02:34 oleg133-client sshd[13474]: Disconnected from user root 192.168.201.33 port 35932 Aug 15 06:02:34 oleg133-client sshd[13461]: pam_unix(sshd:session): session closed for user root Aug 15 06:02:34 oleg133-client systemd[1]: session-41.scope: Succeeded. Aug 15 06:02:34 oleg133-client systemd-logind[1034]: Session 41 logged out. Waiting for processes to exit. Aug 15 06:02:34 oleg133-client systemd-logind[1034]: Removed session 41. Aug 15 06:02:36 oleg133-client sshd[13598]: Accepted publickey for root from 192.168.201.33 port 49124 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:02:36 oleg133-client systemd[1]: Started Session 42 of user root. Aug 15 06:02:36 oleg133-client systemd-logind[1034]: New session 42 of user root. Aug 15 06:02:36 oleg133-client sshd[13598]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:02:36 oleg133-client sshd[13611]: Received disconnect from 192.168.201.33 port 49124:11: disconnected by user Aug 15 06:02:36 oleg133-client sshd[13611]: Disconnected from user root 192.168.201.33 port 49124 Aug 15 06:02:36 oleg133-client sshd[13598]: pam_unix(sshd:session): session closed for user root Aug 15 06:02:36 oleg133-client systemd[1]: session-42.scope: Succeeded. Aug 15 06:02:36 oleg133-client systemd-logind[1034]: Session 42 logged out. Waiting for processes to exit. Aug 15 06:02:36 oleg133-client systemd-logind[1034]: Removed session 42. Aug 15 06:02:37 oleg133-client sshd[13742]: Accepted publickey for root from 192.168.201.33 port 49132 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:02:37 oleg133-client systemd-logind[1034]: New session 43 of user root. Aug 15 06:02:37 oleg133-client systemd[1]: Started Session 43 of user root. Aug 15 06:02:37 oleg133-client sshd[13742]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:02:38 oleg133-client sshd[13756]: Received disconnect from 192.168.201.33 port 49132:11: disconnected by user Aug 15 06:02:38 oleg133-client sshd[13756]: Disconnected from user root 192.168.201.33 port 49132 Aug 15 06:02:38 oleg133-client sshd[13742]: pam_unix(sshd:session): session closed for user root Aug 15 06:02:38 oleg133-client systemd[1]: session-43.scope: Succeeded. Aug 15 06:02:38 oleg133-client systemd-logind[1034]: Session 43 logged out. Waiting for processes to exit. Aug 15 06:02:38 oleg133-client systemd-logind[1034]: Removed session 43. Aug 15 06:02:39 oleg133-client sshd[13896]: Accepted publickey for root from 192.168.201.33 port 49138 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:02:39 oleg133-client systemd[1]: Started Session 44 of user root. Aug 15 06:02:39 oleg133-client systemd-logind[1034]: New session 44 of user root. Aug 15 06:02:39 oleg133-client sshd[13896]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:02:40 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 10a: finish request on server after client eviction (bug 1521) ========================================================== 06:02:38 (1786788158) Aug 15 06:02:40 oleg133-client sshd[13909]: Received disconnect from 192.168.201.33 port 49138:11: disconnected by user Aug 15 06:02:40 oleg133-client sshd[13909]: Disconnected from user root 192.168.201.33 port 49138 Aug 15 06:02:40 oleg133-client sshd[13896]: pam_unix(sshd:session): session closed for user root Aug 15 06:02:40 oleg133-client systemd[1]: session-44.scope: Succeeded. Aug 15 06:02:40 oleg133-client systemd-logind[1034]: Session 44 logged out. Waiting for processes to exit. Aug 15 06:02:40 oleg133-client systemd-logind[1034]: Removed session 44. Aug 15 06:02:40 oleg133-client kernel: Lustre: *** cfs_fail_loc=305, val=0*** Aug 15 06:02:57 oleg133-client kernel: Lustre: lustre-OST0001-osc-ffff993747ebd000: disconnect after 20s idle Aug 15 06:02:57 oleg133-client kernel: Lustre: *** cfs_fail_loc=305, val=0*** Aug 15 06:03:12 oleg133-client kernel: Lustre: *** cfs_fail_loc=305, val=0*** Aug 15 06:03:29 oleg133-client kernel: Lustre: *** cfs_fail_loc=305, val=0*** Aug 15 06:03:45 oleg133-client kernel: Lustre: *** cfs_fail_loc=305, val=0*** Aug 15 06:04:01 oleg133-client kernel: Lustre: *** cfs_fail_loc=305, val=0*** Aug 15 06:04:18 oleg133-client kernel: Lustre: *** cfs_fail_loc=305, val=0*** Aug 15 06:04:33 oleg133-client kernel: LustreError: lustre-MDT0000-mdc-ffff993747ebd000: operation ldlm_enqueue to node 192.168.201.133@tcp failed: rc = -107 Aug 15 06:04:33 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 06:04:33 oleg133-client kernel: Lustre: Skipped 1 previous similar message Aug 15 06:04:33 oleg133-client kernel: LustreError: lustre-MDT0000-mdc-ffff993747ebd000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. Aug 15 06:04:34 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 06:04:34 oleg133-client kernel: Lustre: Skipped 1 previous similar message Aug 15 06:04:38 oleg133-client sshd[14150]: Accepted publickey for root from 192.168.201.33 port 60468 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:04:38 oleg133-client systemd-logind[1034]: New session 45 of user root. Aug 15 06:04:38 oleg133-client systemd[1]: Started Session 45 of user root. Aug 15 06:04:38 oleg133-client sshd[14150]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:04:38 oleg133-client sshd[14163]: Received disconnect from 192.168.201.33 port 60468:11: disconnected by user Aug 15 06:04:38 oleg133-client sshd[14163]: Disconnected from user root 192.168.201.33 port 60468 Aug 15 06:04:38 oleg133-client sshd[14150]: pam_unix(sshd:session): session closed for user root Aug 15 06:04:38 oleg133-client systemd[1]: session-45.scope: Succeeded. Aug 15 06:04:38 oleg133-client systemd-logind[1034]: Session 45 logged out. Waiting for processes to exit. Aug 15 06:04:38 oleg133-client systemd-logind[1034]: Removed session 45. Aug 15 06:04:39 oleg133-client sshd[14288]: Accepted publickey for root from 192.168.201.33 port 60470 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:04:39 oleg133-client systemd[1]: Started Session 46 of user root. Aug 15 06:04:39 oleg133-client systemd-logind[1034]: New session 46 of user root. Aug 15 06:04:39 oleg133-client sshd[14288]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:04:39 oleg133-client sshd[14301]: Received disconnect from 192.168.201.33 port 60470:11: disconnected by user Aug 15 06:04:39 oleg133-client sshd[14301]: Disconnected from user root 192.168.201.33 port 60470 Aug 15 06:04:39 oleg133-client sshd[14288]: pam_unix(sshd:session): session closed for user root Aug 15 06:04:39 oleg133-client systemd[1]: session-46.scope: Succeeded. Aug 15 06:04:39 oleg133-client systemd-logind[1034]: Session 46 logged out. Waiting for processes to exit. Aug 15 06:04:39 oleg133-client systemd-logind[1034]: Removed session 46. Aug 15 06:04:40 oleg133-client sshd[14432]: Accepted publickey for root from 192.168.201.33 port 60472 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:04:40 oleg133-client systemd[1]: Started Session 47 of user root. Aug 15 06:04:40 oleg133-client systemd-logind[1034]: New session 47 of user root. Aug 15 06:04:40 oleg133-client sshd[14432]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:04:41 oleg133-client sshd[14445]: Received disconnect from 192.168.201.33 port 60472:11: disconnected by user Aug 15 06:04:41 oleg133-client sshd[14445]: Disconnected from user root 192.168.201.33 port 60472 Aug 15 06:04:41 oleg133-client sshd[14432]: pam_unix(sshd:session): session closed for user root Aug 15 06:04:41 oleg133-client systemd[1]: session-47.scope: Succeeded. Aug 15 06:04:41 oleg133-client systemd-logind[1034]: Session 47 logged out. Waiting for processes to exit. Aug 15 06:04:41 oleg133-client systemd-logind[1034]: Removed session 47. Aug 15 06:04:42 oleg133-client sshd[14586]: Accepted publickey for root from 192.168.201.33 port 60480 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:04:42 oleg133-client systemd[1]: Started Session 48 of user root. Aug 15 06:04:42 oleg133-client systemd-logind[1034]: New session 48 of user root. Aug 15 06:04:42 oleg133-client sshd[14586]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:04:42 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 10b: re-send BL AST =============== 06:04:41 (1786788281) Aug 15 06:04:43 oleg133-client sshd[14599]: Received disconnect from 192.168.201.33 port 60480:11: disconnected by user Aug 15 06:04:43 oleg133-client sshd[14599]: Disconnected from user root 192.168.201.33 port 60480 Aug 15 06:04:43 oleg133-client sshd[14586]: pam_unix(sshd:session): session closed for user root Aug 15 06:04:43 oleg133-client systemd[1]: session-48.scope: Succeeded. Aug 15 06:04:43 oleg133-client systemd-logind[1034]: Session 48 logged out. Waiting for processes to exit. Aug 15 06:04:43 oleg133-client systemd-logind[1034]: Removed session 48. Aug 15 06:04:59 oleg133-client kernel: Lustre: lustre-OST0001-osc-ffff993747ebd000: disconnect after 24s idle Aug 15 06:05:01 oleg133-client sshd[14793]: Accepted publickey for root from 192.168.201.33 port 48030 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:01 oleg133-client systemd[1]: Started Session 49 of user root. Aug 15 06:05:01 oleg133-client systemd-logind[1034]: New session 49 of user root. Aug 15 06:05:01 oleg133-client sshd[14793]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:02 oleg133-client sshd[14806]: Received disconnect from 192.168.201.33 port 48030:11: disconnected by user Aug 15 06:05:02 oleg133-client sshd[14806]: Disconnected from user root 192.168.201.33 port 48030 Aug 15 06:05:02 oleg133-client sshd[14793]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:02 oleg133-client systemd[1]: session-49.scope: Succeeded. Aug 15 06:05:02 oleg133-client systemd-logind[1034]: Session 49 logged out. Waiting for processes to exit. Aug 15 06:05:02 oleg133-client systemd-logind[1034]: Removed session 49. Aug 15 06:05:04 oleg133-client sshd[14930]: Accepted publickey for root from 192.168.201.33 port 48032 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:04 oleg133-client systemd-logind[1034]: New session 50 of user root. Aug 15 06:05:04 oleg133-client systemd[1]: Started Session 50 of user root. Aug 15 06:05:04 oleg133-client sshd[14930]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:04 oleg133-client sshd[14943]: Received disconnect from 192.168.201.33 port 48032:11: disconnected by user Aug 15 06:05:04 oleg133-client sshd[14943]: Disconnected from user root 192.168.201.33 port 48032 Aug 15 06:05:04 oleg133-client sshd[14930]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:04 oleg133-client systemd[1]: session-50.scope: Succeeded. Aug 15 06:05:04 oleg133-client systemd-logind[1034]: Session 50 logged out. Waiting for processes to exit. Aug 15 06:05:04 oleg133-client systemd-logind[1034]: Removed session 50. Aug 15 06:05:05 oleg133-client sshd[15074]: Accepted publickey for root from 192.168.201.33 port 52858 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:06 oleg133-client systemd-logind[1034]: New session 51 of user root. Aug 15 06:05:06 oleg133-client systemd[1]: Started Session 51 of user root. Aug 15 06:05:06 oleg133-client sshd[15074]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:06 oleg133-client sshd[15088]: Received disconnect from 192.168.201.33 port 52858:11: disconnected by user Aug 15 06:05:06 oleg133-client sshd[15088]: Disconnected from user root 192.168.201.33 port 52858 Aug 15 06:05:06 oleg133-client sshd[15074]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:06 oleg133-client systemd[1]: session-51.scope: Succeeded. Aug 15 06:05:06 oleg133-client systemd-logind[1034]: Session 51 logged out. Waiting for processes to exit. Aug 15 06:05:06 oleg133-client systemd-logind[1034]: Removed session 51. Aug 15 06:05:08 oleg133-client sshd[15228]: Accepted publickey for root from 192.168.201.33 port 52870 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:08 oleg133-client systemd[1]: Started Session 52 of user root. Aug 15 06:05:08 oleg133-client systemd-logind[1034]: New session 52 of user root. Aug 15 06:05:08 oleg133-client sshd[15228]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:09 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 10c: re-send BL AST vs reconnect race (LU-5569) ========================================================== 06:05:07 (1786788307) Aug 15 06:05:09 oleg133-client sshd[15241]: Received disconnect from 192.168.201.33 port 52870:11: disconnected by user Aug 15 06:05:09 oleg133-client sshd[15241]: Disconnected from user root 192.168.201.33 port 52870 Aug 15 06:05:09 oleg133-client sshd[15228]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:09 oleg133-client systemd[1]: session-52.scope: Succeeded. Aug 15 06:05:09 oleg133-client systemd-logind[1034]: Session 52 logged out. Waiting for processes to exit. Aug 15 06:05:09 oleg133-client systemd-logind[1034]: Removed session 52. Aug 15 06:05:09 oleg133-client kernel: Lustre: *** cfs_fail_loc=305, val=0*** Aug 15 06:05:09 oleg133-client kernel: Lustre: Skipped 1 previous similar message Aug 15 06:05:12 oleg133-client sshd[15428]: Accepted publickey for root from 192.168.201.33 port 52882 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:13 oleg133-client systemd-logind[1034]: New session 53 of user root. Aug 15 06:05:13 oleg133-client systemd[1]: Started Session 53 of user root. Aug 15 06:05:13 oleg133-client sshd[15428]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:13 oleg133-client sshd[15441]: Received disconnect from 192.168.201.33 port 52882:11: disconnected by user Aug 15 06:05:13 oleg133-client sshd[15441]: Disconnected from user root 192.168.201.33 port 52882 Aug 15 06:05:13 oleg133-client sshd[15428]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:13 oleg133-client systemd-logind[1034]: Session 53 logged out. Waiting for processes to exit. Aug 15 06:05:13 oleg133-client systemd[1]: session-53.scope: Succeeded. Aug 15 06:05:13 oleg133-client systemd-logind[1034]: Removed session 53. Aug 15 06:05:14 oleg133-client sshd[15565]: Accepted publickey for root from 192.168.201.33 port 52892 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:14 oleg133-client systemd-logind[1034]: New session 54 of user root. Aug 15 06:05:14 oleg133-client systemd[1]: Started Session 54 of user root. Aug 15 06:05:14 oleg133-client sshd[15565]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:15 oleg133-client sshd[15578]: Received disconnect from 192.168.201.33 port 52892:11: disconnected by user Aug 15 06:05:15 oleg133-client sshd[15578]: Disconnected from user root 192.168.201.33 port 52892 Aug 15 06:05:15 oleg133-client sshd[15565]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:15 oleg133-client systemd[1]: session-54.scope: Succeeded. Aug 15 06:05:15 oleg133-client systemd-logind[1034]: Session 54 logged out. Waiting for processes to exit. Aug 15 06:05:15 oleg133-client systemd-logind[1034]: Removed session 54. Aug 15 06:05:16 oleg133-client sshd[15709]: Accepted publickey for root from 192.168.201.33 port 53606 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:16 oleg133-client systemd-logind[1034]: New session 55 of user root. Aug 15 06:05:16 oleg133-client systemd[1]: Started Session 55 of user root. Aug 15 06:05:16 oleg133-client sshd[15709]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:16 oleg133-client sshd[15722]: Received disconnect from 192.168.201.33 port 53606:11: disconnected by user Aug 15 06:05:16 oleg133-client sshd[15722]: Disconnected from user root 192.168.201.33 port 53606 Aug 15 06:05:16 oleg133-client sshd[15709]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:17 oleg133-client systemd[1]: session-55.scope: Succeeded. Aug 15 06:05:17 oleg133-client systemd-logind[1034]: Session 55 logged out. Waiting for processes to exit. Aug 15 06:05:17 oleg133-client systemd-logind[1034]: Removed session 55. Aug 15 06:05:18 oleg133-client sshd[15863]: Accepted publickey for root from 192.168.201.33 port 53610 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:18 oleg133-client systemd-logind[1034]: New session 56 of user root. Aug 15 06:05:18 oleg133-client systemd[1]: Started Session 56 of user root. Aug 15 06:05:18 oleg133-client sshd[15863]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:18 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 10d: test failed blocking ast ===== 06:05:17 (1786788317) Aug 15 06:05:18 oleg133-client sshd[15876]: Received disconnect from 192.168.201.33 port 53610:11: disconnected by user Aug 15 06:05:18 oleg133-client sshd[15876]: Disconnected from user root 192.168.201.33 port 53610 Aug 15 06:05:18 oleg133-client sshd[15863]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:18 oleg133-client systemd[1]: session-56.scope: Succeeded. Aug 15 06:05:18 oleg133-client systemd-logind[1034]: Session 56 logged out. Waiting for processes to exit. Aug 15 06:05:18 oleg133-client systemd-logind[1034]: Removed session 56. Aug 15 06:05:22 oleg133-client systemd[1]: mnt-lustre.mount: Succeeded. Aug 15 06:05:22 oleg133-client kernel: Lustre: Unmounted lustre-client Aug 15 06:05:22 oleg133-client systemd-udevd[15972]: Process '/home/green/git/lustre-release/lustre/utils/lctl set_param 'lod.*.mdt_hash=crush'' failed with exit code 2. Aug 15 06:05:22 oleg133-client kernel: Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 Aug 15 06:05:23 oleg133-client systemd-udevd[16004]: Process '/home/green/git/lustre-release/lustre/utils/lctl set_param 'lod.*.mdt_hash=crush'' failed with exit code 2. Aug 15 06:05:24 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: operation ldlm_enqueue to node 192.168.201.133@tcp failed: rc = -107 Aug 15 06:05:24 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. Aug 15 06:05:24 oleg133-client kernel: Lustre: 2376:0:(llite_lib.c:4340:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.201.133@tcp:/lustre/fid: [0x200000403:0x1:0x0]/ may get corrupted (rc -108) Aug 15 06:05:27 oleg133-client systemd[1]: mnt-lustre2.mount: Succeeded. Aug 15 06:05:27 oleg133-client kernel: Lustre: Unmounted lustre-client Aug 15 06:05:28 oleg133-client sshd[16204]: Accepted publickey for root from 192.168.201.33 port 49188 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:28 oleg133-client systemd-logind[1034]: New session 57 of user root. Aug 15 06:05:28 oleg133-client systemd[1]: Started Session 57 of user root. Aug 15 06:05:28 oleg133-client sshd[16204]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:29 oleg133-client sshd[16218]: Received disconnect from 192.168.201.33 port 49188:11: disconnected by user Aug 15 06:05:29 oleg133-client sshd[16218]: Disconnected from user root 192.168.201.33 port 49188 Aug 15 06:05:29 oleg133-client sshd[16204]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:29 oleg133-client systemd[1]: session-57.scope: Succeeded. Aug 15 06:05:29 oleg133-client systemd-logind[1034]: Session 57 logged out. Waiting for processes to exit. Aug 15 06:05:29 oleg133-client systemd-logind[1034]: Removed session 57. Aug 15 06:05:30 oleg133-client sshd[16342]: Accepted publickey for root from 192.168.201.33 port 49192 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:30 oleg133-client systemd-logind[1034]: New session 58 of user root. Aug 15 06:05:30 oleg133-client systemd[1]: Started Session 58 of user root. Aug 15 06:05:30 oleg133-client sshd[16342]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:30 oleg133-client sshd[16355]: Received disconnect from 192.168.201.33 port 49192:11: disconnected by user Aug 15 06:05:30 oleg133-client sshd[16355]: Disconnected from user root 192.168.201.33 port 49192 Aug 15 06:05:30 oleg133-client sshd[16342]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:31 oleg133-client systemd-logind[1034]: Session 58 logged out. Waiting for processes to exit. Aug 15 06:05:31 oleg133-client systemd[1]: session-58.scope: Succeeded. Aug 15 06:05:31 oleg133-client systemd-logind[1034]: Removed session 58. Aug 15 06:05:32 oleg133-client sshd[16486]: Accepted publickey for root from 192.168.201.33 port 49204 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:32 oleg133-client systemd[1]: Started Session 59 of user root. Aug 15 06:05:32 oleg133-client systemd-logind[1034]: New session 59 of user root. Aug 15 06:05:32 oleg133-client sshd[16486]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:32 oleg133-client sshd[16499]: Received disconnect from 192.168.201.33 port 49204:11: disconnected by user Aug 15 06:05:32 oleg133-client sshd[16499]: Disconnected from user root 192.168.201.33 port 49204 Aug 15 06:05:32 oleg133-client sshd[16486]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:32 oleg133-client systemd-logind[1034]: Session 59 logged out. Waiting for processes to exit. Aug 15 06:05:32 oleg133-client systemd[1]: session-59.scope: Succeeded. Aug 15 06:05:32 oleg133-client systemd-logind[1034]: Removed session 59. Aug 15 06:05:34 oleg133-client sshd[16639]: Accepted publickey for root from 192.168.201.33 port 49210 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:34 oleg133-client systemd[1]: Started Session 60 of user root. Aug 15 06:05:34 oleg133-client systemd-logind[1034]: New session 60 of user root. Aug 15 06:05:34 oleg133-client sshd[16639]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:34 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 10e: re-send BL AST vs reconnect race 2 ========================================================== 06:05:32 (1786788332) Aug 15 06:05:34 oleg133-client sshd[16653]: Received disconnect from 192.168.201.33 port 49210:11: disconnected by user Aug 15 06:05:34 oleg133-client sshd[16653]: Disconnected from user root 192.168.201.33 port 49210 Aug 15 06:05:34 oleg133-client sshd[16639]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:35 oleg133-client systemd[1]: session-60.scope: Succeeded. Aug 15 06:05:35 oleg133-client systemd-logind[1034]: Session 60 logged out. Waiting for processes to exit. Aug 15 06:05:35 oleg133-client systemd-logind[1034]: Removed session 60. Aug 15 06:05:35 oleg133-client sshd[16779]: Accepted publickey for root from 192.168.201.33 port 44698 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:35 oleg133-client systemd[1]: Started Session 61 of user root. Aug 15 06:05:35 oleg133-client systemd-logind[1034]: New session 61 of user root. Aug 15 06:05:35 oleg133-client sshd[16779]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:36 oleg133-client kernel: Lustre: DEBUG MARKER: SKIP: recovery-small test_10e need two clients Aug 15 06:05:36 oleg133-client sshd[16792]: Received disconnect from 192.168.201.33 port 44698:11: disconnected by user Aug 15 06:05:36 oleg133-client sshd[16792]: Disconnected from user root 192.168.201.33 port 44698 Aug 15 06:05:36 oleg133-client sshd[16779]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:36 oleg133-client systemd-logind[1034]: Session 61 logged out. Waiting for processes to exit. Aug 15 06:05:36 oleg133-client systemd[1]: session-61.scope: Succeeded. Aug 15 06:05:36 oleg133-client systemd-logind[1034]: Removed session 61. Aug 15 06:05:37 oleg133-client sshd[16931]: Accepted publickey for root from 192.168.201.33 port 44700 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:37 oleg133-client systemd-logind[1034]: New session 62 of user root. Aug 15 06:05:37 oleg133-client systemd[1]: Started Session 62 of user root. Aug 15 06:05:37 oleg133-client sshd[16931]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:38 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 11: wake up a thread waiting for completion after eviction (b=2460) ========================================================== 06:05:36 (1786788336) Aug 15 06:05:38 oleg133-client sshd[16944]: Received disconnect from 192.168.201.33 port 44700:11: disconnected by user Aug 15 06:05:38 oleg133-client sshd[16944]: Disconnected from user root 192.168.201.33 port 44700 Aug 15 06:05:38 oleg133-client sshd[16931]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:38 oleg133-client systemd-logind[1034]: Session 62 logged out. Waiting for processes to exit. Aug 15 06:05:38 oleg133-client systemd[1]: session-62.scope: Succeeded. Aug 15 06:05:38 oleg133-client systemd-logind[1034]: Removed session 62. Aug 15 06:05:57 oleg133-client sshd[17053]: Accepted publickey for root from 192.168.201.33 port 37434 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:57 oleg133-client systemd[1]: Started Session 63 of user root. Aug 15 06:05:57 oleg133-client systemd-logind[1034]: New session 63 of user root. Aug 15 06:05:57 oleg133-client sshd[17053]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:57 oleg133-client sshd[17066]: Received disconnect from 192.168.201.33 port 37434:11: disconnected by user Aug 15 06:05:57 oleg133-client sshd[17066]: Disconnected from user root 192.168.201.33 port 37434 Aug 15 06:05:57 oleg133-client sshd[17053]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:57 oleg133-client systemd[1]: session-63.scope: Succeeded. Aug 15 06:05:57 oleg133-client systemd-logind[1034]: Session 63 logged out. Waiting for processes to exit. Aug 15 06:05:57 oleg133-client systemd-logind[1034]: Removed session 63. Aug 15 06:05:58 oleg133-client sshd[17184]: Accepted publickey for root from 192.168.201.33 port 37444 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:05:58 oleg133-client systemd[1]: Started Session 64 of user root. Aug 15 06:05:58 oleg133-client systemd-logind[1034]: New session 64 of user root. Aug 15 06:05:58 oleg133-client sshd[17184]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:05:58 oleg133-client sshd[17197]: Received disconnect from 192.168.201.33 port 37444:11: disconnected by user Aug 15 06:05:58 oleg133-client sshd[17197]: Disconnected from user root 192.168.201.33 port 37444 Aug 15 06:05:58 oleg133-client sshd[17184]: pam_unix(sshd:session): session closed for user root Aug 15 06:05:58 oleg133-client systemd[1]: session-64.scope: Succeeded. Aug 15 06:05:58 oleg133-client systemd-logind[1034]: Session 64 logged out. Waiting for processes to exit. Aug 15 06:05:58 oleg133-client systemd-logind[1034]: Removed session 64. Aug 15 06:06:00 oleg133-client sshd[17322]: Accepted publickey for root from 192.168.201.33 port 37456 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:06:01 oleg133-client systemd-logind[1034]: New session 65 of user root. Aug 15 06:06:01 oleg133-client systemd[1]: Started Session 65 of user root. Aug 15 06:06:01 oleg133-client sshd[17322]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:06:01 oleg133-client sshd[17335]: Received disconnect from 192.168.201.33 port 37456:11: disconnected by user Aug 15 06:06:01 oleg133-client sshd[17335]: Disconnected from user root 192.168.201.33 port 37456 Aug 15 06:06:01 oleg133-client sshd[17322]: pam_unix(sshd:session): session closed for user root Aug 15 06:06:01 oleg133-client systemd[1]: session-65.scope: Succeeded. Aug 15 06:06:01 oleg133-client systemd-logind[1034]: Session 65 logged out. Waiting for processes to exit. Aug 15 06:06:01 oleg133-client systemd-logind[1034]: Removed session 65. Aug 15 06:06:02 oleg133-client sshd[17466]: Accepted publickey for root from 192.168.201.33 port 37472 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:06:02 oleg133-client systemd[1]: Started Session 66 of user root. Aug 15 06:06:02 oleg133-client systemd-logind[1034]: New session 66 of user root. Aug 15 06:06:02 oleg133-client sshd[17466]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:06:02 oleg133-client sshd[17479]: Received disconnect from 192.168.201.33 port 37472:11: disconnected by user Aug 15 06:06:02 oleg133-client sshd[17479]: Disconnected from user root 192.168.201.33 port 37472 Aug 15 06:06:02 oleg133-client sshd[17466]: pam_unix(sshd:session): session closed for user root Aug 15 06:06:02 oleg133-client systemd[1]: session-66.scope: Succeeded. Aug 15 06:06:02 oleg133-client systemd-logind[1034]: Session 66 logged out. Waiting for processes to exit. Aug 15 06:06:02 oleg133-client systemd-logind[1034]: Removed session 66. Aug 15 06:06:04 oleg133-client sshd[17619]: Accepted publickey for root from 192.168.201.33 port 37476 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:06:04 oleg133-client systemd-logind[1034]: New session 67 of user root. Aug 15 06:06:04 oleg133-client systemd[1]: Started Session 67 of user root. Aug 15 06:06:04 oleg133-client sshd[17619]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:06:04 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 12: recover from timed out resend in ptlrpcd (b=2494) ========================================================== 06:06:03 (1786788363) Aug 15 06:06:04 oleg133-client sshd[17632]: Received disconnect from 192.168.201.33 port 37476:11: disconnected by user Aug 15 06:06:04 oleg133-client sshd[17632]: Disconnected from user root 192.168.201.33 port 37476 Aug 15 06:06:04 oleg133-client sshd[17619]: pam_unix(sshd:session): session closed for user root Aug 15 06:06:05 oleg133-client kernel: Lustre: DEBUG MARKER: multiop /mnt/lustre/f12.recovery-small OS_c Aug 15 06:06:05 oleg133-client systemd-logind[1034]: Session 67 logged out. Waiting for processes to exit. Aug 15 06:06:05 oleg133-client systemd[1]: session-67.scope: Succeeded. Aug 15 06:06:05 oleg133-client systemd-logind[1034]: Removed session 67. Aug 15 06:06:22 oleg133-client kernel: Lustre: 17679:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786788366/real 1786788366] req@ffff993760045880 x1873582575693824/t0(0) o35->lustre-MDT0000-mdc-ffff993744fd2000@192.168.201.133@tcp:23/10 lens 392/624 e 0 to 1 dl 1786788382 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 Aug 15 06:06:22 oleg133-client kernel: Lustre: 17679:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message Aug 15 06:06:24 oleg133-client sshd[17697]: Accepted publickey for root from 192.168.201.33 port 59152 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:06:24 oleg133-client systemd-logind[1034]: New session 68 of user root. Aug 15 06:06:24 oleg133-client systemd[1]: Started Session 68 of user root. Aug 15 06:06:24 oleg133-client sshd[17697]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:06:24 oleg133-client sshd[17710]: Received disconnect from 192.168.201.33 port 59152:11: disconnected by user Aug 15 06:06:24 oleg133-client sshd[17710]: Disconnected from user root 192.168.201.33 port 59152 Aug 15 06:06:24 oleg133-client sshd[17697]: pam_unix(sshd:session): session closed for user root Aug 15 06:06:25 oleg133-client systemd[1]: session-68.scope: Succeeded. Aug 15 06:06:25 oleg133-client systemd-logind[1034]: Session 68 logged out. Waiting for processes to exit. Aug 15 06:06:25 oleg133-client systemd-logind[1034]: Removed session 68. Aug 15 06:06:25 oleg133-client sshd[17828]: Accepted publickey for root from 192.168.201.33 port 42898 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:06:25 oleg133-client systemd-logind[1034]: New session 69 of user root. Aug 15 06:06:25 oleg133-client systemd[1]: Started Session 69 of user root. Aug 15 06:06:25 oleg133-client sshd[17828]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:06:26 oleg133-client sshd[17841]: Received disconnect from 192.168.201.33 port 42898:11: disconnected by user Aug 15 06:06:26 oleg133-client sshd[17841]: Disconnected from user root 192.168.201.33 port 42898 Aug 15 06:06:26 oleg133-client sshd[17828]: pam_unix(sshd:session): session closed for user root Aug 15 06:06:26 oleg133-client systemd-logind[1034]: Session 69 logged out. Waiting for processes to exit. Aug 15 06:06:26 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:06:26 oleg133-client systemd[1]: session-69.scope: Succeeded. Aug 15 06:06:26 oleg133-client systemd-logind[1034]: Removed session 69. Aug 15 06:06:28 oleg133-client sshd[17966]: Accepted publickey for root from 192.168.201.33 port 42904 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:06:28 oleg133-client systemd-logind[1034]: New session 70 of user root. Aug 15 06:06:28 oleg133-client systemd[1]: Started Session 70 of user root. Aug 15 06:06:28 oleg133-client sshd[17966]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:06:28 oleg133-client sshd[17979]: Received disconnect from 192.168.201.33 port 42904:11: disconnected by user Aug 15 06:06:28 oleg133-client sshd[17979]: Disconnected from user root 192.168.201.33 port 42904 Aug 15 06:06:28 oleg133-client sshd[17966]: pam_unix(sshd:session): session closed for user root Aug 15 06:06:29 oleg133-client systemd[1]: session-70.scope: Succeeded. Aug 15 06:06:29 oleg133-client systemd-logind[1034]: Session 70 logged out. Waiting for processes to exit. Aug 15 06:06:29 oleg133-client systemd-logind[1034]: Removed session 70. Aug 15 06:06:29 oleg133-client sshd[18110]: Accepted publickey for root from 192.168.201.33 port 42906 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:06:29 oleg133-client systemd-logind[1034]: New session 71 of user root. Aug 15 06:06:29 oleg133-client systemd[1]: Started Session 71 of user root. Aug 15 06:06:29 oleg133-client sshd[18110]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:06:30 oleg133-client sshd[18123]: Received disconnect from 192.168.201.33 port 42906:11: disconnected by user Aug 15 06:06:30 oleg133-client sshd[18123]: Disconnected from user root 192.168.201.33 port 42906 Aug 15 06:06:30 oleg133-client sshd[18110]: pam_unix(sshd:session): session closed for user root Aug 15 06:06:30 oleg133-client systemd[1]: session-71.scope: Succeeded. Aug 15 06:06:30 oleg133-client systemd-logind[1034]: Session 71 logged out. Waiting for processes to exit. Aug 15 06:06:30 oleg133-client systemd-logind[1034]: Removed session 71. Aug 15 06:06:50 oleg133-client sshd[18274]: Accepted publickey for root from 192.168.201.33 port 57274 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:06:50 oleg133-client systemd[1]: Started Session 72 of user root. Aug 15 06:06:50 oleg133-client systemd-logind[1034]: New session 72 of user root. Aug 15 06:06:50 oleg133-client sshd[18274]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:06:50 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 13: mdc_readpage restart test (bug 1138) ========================================================== 06:06:48 (1786788408) Aug 15 06:06:51 oleg133-client sshd[18287]: Received disconnect from 192.168.201.33 port 57274:11: disconnected by user Aug 15 06:06:51 oleg133-client sshd[18287]: Disconnected from user root 192.168.201.33 port 57274 Aug 15 06:06:51 oleg133-client sshd[18274]: pam_unix(sshd:session): session closed for user root Aug 15 06:06:51 oleg133-client systemd-logind[1034]: Session 72 logged out. Waiting for processes to exit. Aug 15 06:06:51 oleg133-client systemd[1]: session-72.scope: Succeeded. Aug 15 06:06:51 oleg133-client systemd-logind[1034]: Removed session 72. Aug 15 06:06:54 oleg133-client systemd[1]: Starting Cleanup of Temporary Directories... Aug 15 06:06:55 oleg133-client systemd[1]: systemd-tmpfiles-clean.service: Succeeded. Aug 15 06:06:55 oleg133-client systemd[1]: Started Cleanup of Temporary Directories. Aug 15 06:07:11 oleg133-client sshd[18432]: Accepted publickey for root from 192.168.201.33 port 45024 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:11 oleg133-client systemd-logind[1034]: New session 73 of user root. Aug 15 06:07:11 oleg133-client systemd[1]: Started Session 73 of user root. Aug 15 06:07:11 oleg133-client sshd[18432]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:11 oleg133-client sshd[18446]: Received disconnect from 192.168.201.33 port 45024:11: disconnected by user Aug 15 06:07:11 oleg133-client sshd[18446]: Disconnected from user root 192.168.201.33 port 45024 Aug 15 06:07:11 oleg133-client sshd[18432]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:12 oleg133-client systemd-logind[1034]: Session 73 logged out. Waiting for processes to exit. Aug 15 06:07:12 oleg133-client systemd[1]: session-73.scope: Succeeded. Aug 15 06:07:12 oleg133-client systemd-logind[1034]: Removed session 73. Aug 15 06:07:13 oleg133-client sshd[18570]: Accepted publickey for root from 192.168.201.33 port 45026 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:13 oleg133-client systemd[1]: Started Session 74 of user root. Aug 15 06:07:13 oleg133-client systemd-logind[1034]: New session 74 of user root. Aug 15 06:07:13 oleg133-client sshd[18570]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:13 oleg133-client sshd[18583]: Received disconnect from 192.168.201.33 port 45026:11: disconnected by user Aug 15 06:07:13 oleg133-client sshd[18583]: Disconnected from user root 192.168.201.33 port 45026 Aug 15 06:07:13 oleg133-client sshd[18570]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:14 oleg133-client systemd-logind[1034]: Session 74 logged out. Waiting for processes to exit. Aug 15 06:07:14 oleg133-client systemd[1]: session-74.scope: Succeeded. Aug 15 06:07:14 oleg133-client systemd-logind[1034]: Removed session 74. Aug 15 06:07:15 oleg133-client sshd[18714]: Accepted publickey for root from 192.168.201.33 port 34486 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:15 oleg133-client systemd-logind[1034]: New session 75 of user root. Aug 15 06:07:15 oleg133-client systemd[1]: Started Session 75 of user root. Aug 15 06:07:15 oleg133-client sshd[18714]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:16 oleg133-client sshd[18727]: Received disconnect from 192.168.201.33 port 34486:11: disconnected by user Aug 15 06:07:16 oleg133-client sshd[18727]: Disconnected from user root 192.168.201.33 port 34486 Aug 15 06:07:16 oleg133-client sshd[18714]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:16 oleg133-client systemd[1]: session-75.scope: Succeeded. Aug 15 06:07:16 oleg133-client systemd-logind[1034]: Session 75 logged out. Waiting for processes to exit. Aug 15 06:07:16 oleg133-client systemd-logind[1034]: Removed session 75. Aug 15 06:07:18 oleg133-client sshd[18868]: Accepted publickey for root from 192.168.201.33 port 34496 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:18 oleg133-client systemd-logind[1034]: New session 76 of user root. Aug 15 06:07:18 oleg133-client systemd[1]: Started Session 76 of user root. Aug 15 06:07:18 oleg133-client sshd[18868]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:18 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 14: mdc_readpage resend test (bug 1138) ========================================================== 06:07:16 (1786788436) Aug 15 06:07:18 oleg133-client sshd[18881]: Received disconnect from 192.168.201.33 port 34496:11: disconnected by user Aug 15 06:07:18 oleg133-client sshd[18881]: Disconnected from user root 192.168.201.33 port 34496 Aug 15 06:07:18 oleg133-client sshd[18868]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:18 oleg133-client systemd-logind[1034]: Session 76 logged out. Waiting for processes to exit. Aug 15 06:07:18 oleg133-client systemd[1]: session-76.scope: Succeeded. Aug 15 06:07:18 oleg133-client systemd-logind[1034]: Removed session 76. Aug 15 06:07:22 oleg133-client sshd[19020]: Accepted publickey for root from 192.168.201.33 port 34500 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:22 oleg133-client systemd-logind[1034]: New session 77 of user root. Aug 15 06:07:22 oleg133-client systemd[1]: Started Session 77 of user root. Aug 15 06:07:22 oleg133-client sshd[19020]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:23 oleg133-client sshd[19033]: Received disconnect from 192.168.201.33 port 34500:11: disconnected by user Aug 15 06:07:23 oleg133-client sshd[19033]: Disconnected from user root 192.168.201.33 port 34500 Aug 15 06:07:23 oleg133-client sshd[19020]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:23 oleg133-client systemd-logind[1034]: Session 77 logged out. Waiting for processes to exit. Aug 15 06:07:23 oleg133-client systemd[1]: session-77.scope: Succeeded. Aug 15 06:07:23 oleg133-client systemd-logind[1034]: Removed session 77. Aug 15 06:07:24 oleg133-client sshd[19158]: Accepted publickey for root from 192.168.201.33 port 34502 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:24 oleg133-client systemd-logind[1034]: New session 78 of user root. Aug 15 06:07:24 oleg133-client systemd[1]: Started Session 78 of user root. Aug 15 06:07:24 oleg133-client sshd[19158]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:24 oleg133-client sshd[19171]: Received disconnect from 192.168.201.33 port 34502:11: disconnected by user Aug 15 06:07:24 oleg133-client sshd[19171]: Disconnected from user root 192.168.201.33 port 34502 Aug 15 06:07:24 oleg133-client sshd[19158]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:24 oleg133-client systemd-logind[1034]: Session 78 logged out. Waiting for processes to exit. Aug 15 06:07:24 oleg133-client systemd[1]: session-78.scope: Succeeded. Aug 15 06:07:24 oleg133-client systemd-logind[1034]: Removed session 78. Aug 15 06:07:25 oleg133-client sshd[19303]: Accepted publickey for root from 192.168.201.33 port 59598 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:25 oleg133-client systemd-logind[1034]: New session 79 of user root. Aug 15 06:07:25 oleg133-client systemd[1]: Started Session 79 of user root. Aug 15 06:07:25 oleg133-client sshd[19303]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:26 oleg133-client sshd[19316]: Received disconnect from 192.168.201.33 port 59598:11: disconnected by user Aug 15 06:07:26 oleg133-client sshd[19316]: Disconnected from user root 192.168.201.33 port 59598 Aug 15 06:07:26 oleg133-client sshd[19303]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:26 oleg133-client systemd[1]: session-79.scope: Succeeded. Aug 15 06:07:26 oleg133-client systemd-logind[1034]: Session 79 logged out. Waiting for processes to exit. Aug 15 06:07:26 oleg133-client systemd-logind[1034]: Removed session 79. Aug 15 06:07:27 oleg133-client sshd[19456]: Accepted publickey for root from 192.168.201.33 port 59602 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:27 oleg133-client systemd-logind[1034]: New session 80 of user root. Aug 15 06:07:27 oleg133-client systemd[1]: Started Session 80 of user root. Aug 15 06:07:27 oleg133-client sshd[19456]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:28 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 15: failed open (-ENOMEM) ========= 06:07:26 (1786788446) Aug 15 06:07:28 oleg133-client sshd[19469]: Received disconnect from 192.168.201.33 port 59602:11: disconnected by user Aug 15 06:07:28 oleg133-client sshd[19469]: Disconnected from user root 192.168.201.33 port 59602 Aug 15 06:07:28 oleg133-client sshd[19456]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:28 oleg133-client systemd[1]: session-80.scope: Succeeded. Aug 15 06:07:28 oleg133-client systemd-logind[1034]: Session 80 logged out. Waiting for processes to exit. Aug 15 06:07:28 oleg133-client systemd-logind[1034]: Removed session 80. Aug 15 06:07:30 oleg133-client sshd[19597]: Accepted publickey for root from 192.168.201.33 port 59608 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:30 oleg133-client systemd-logind[1034]: New session 81 of user root. Aug 15 06:07:30 oleg133-client systemd[1]: Started Session 81 of user root. Aug 15 06:07:30 oleg133-client sshd[19597]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:31 oleg133-client sshd[19610]: Received disconnect from 192.168.201.33 port 59608:11: disconnected by user Aug 15 06:07:31 oleg133-client sshd[19610]: Disconnected from user root 192.168.201.33 port 59608 Aug 15 06:07:31 oleg133-client sshd[19597]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:31 oleg133-client systemd[1]: session-81.scope: Succeeded. Aug 15 06:07:31 oleg133-client systemd-logind[1034]: Session 81 logged out. Waiting for processes to exit. Aug 15 06:07:31 oleg133-client systemd-logind[1034]: Removed session 81. Aug 15 06:07:32 oleg133-client sshd[19735]: Accepted publickey for root from 192.168.201.33 port 59618 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:32 oleg133-client systemd-logind[1034]: New session 82 of user root. Aug 15 06:07:32 oleg133-client systemd[1]: Started Session 82 of user root. Aug 15 06:07:32 oleg133-client sshd[19735]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:32 oleg133-client sshd[19748]: Received disconnect from 192.168.201.33 port 59618:11: disconnected by user Aug 15 06:07:32 oleg133-client sshd[19748]: Disconnected from user root 192.168.201.33 port 59618 Aug 15 06:07:32 oleg133-client sshd[19735]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:32 oleg133-client systemd[1]: session-82.scope: Succeeded. Aug 15 06:07:32 oleg133-client systemd-logind[1034]: Session 82 logged out. Waiting for processes to exit. Aug 15 06:07:32 oleg133-client systemd-logind[1034]: Removed session 82. Aug 15 06:07:33 oleg133-client sshd[19879]: Accepted publickey for root from 192.168.201.33 port 59622 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:33 oleg133-client systemd[1]: Started Session 83 of user root. Aug 15 06:07:33 oleg133-client systemd-logind[1034]: New session 83 of user root. Aug 15 06:07:33 oleg133-client sshd[19879]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:33 oleg133-client sshd[19892]: Received disconnect from 192.168.201.33 port 59622:11: disconnected by user Aug 15 06:07:33 oleg133-client sshd[19892]: Disconnected from user root 192.168.201.33 port 59622 Aug 15 06:07:33 oleg133-client sshd[19879]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:33 oleg133-client systemd[1]: session-83.scope: Succeeded. Aug 15 06:07:33 oleg133-client systemd-logind[1034]: Session 83 logged out. Waiting for processes to exit. Aug 15 06:07:33 oleg133-client systemd-logind[1034]: Removed session 83. Aug 15 06:07:34 oleg133-client sshd[20035]: Accepted publickey for root from 192.168.201.33 port 59628 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:07:34 oleg133-client systemd[1]: Started Session 84 of user root. Aug 15 06:07:34 oleg133-client systemd-logind[1034]: New session 84 of user root. Aug 15 06:07:34 oleg133-client sshd[20035]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:07:35 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 16: timeout bulk put, don't evict client (2732) ========================================================== 06:07:33 (1786788453) Aug 15 06:07:35 oleg133-client sshd[20048]: Received disconnect from 192.168.201.33 port 59628:11: disconnected by user Aug 15 06:07:35 oleg133-client sshd[20048]: Disconnected from user root 192.168.201.33 port 59628 Aug 15 06:07:35 oleg133-client sshd[20035]: pam_unix(sshd:session): session closed for user root Aug 15 06:07:35 oleg133-client systemd-logind[1034]: Session 84 logged out. Waiting for processes to exit. Aug 15 06:07:35 oleg133-client systemd[1]: session-84.scope: Succeeded. Aug 15 06:07:35 oleg133-client systemd-logind[1034]: Removed session 84. Aug 15 06:08:15 oleg133-client sshd[20256]: Accepted publickey for root from 192.168.201.33 port 51696 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:08:15 oleg133-client systemd-logind[1034]: New session 85 of user root. Aug 15 06:08:15 oleg133-client systemd[1]: Started Session 85 of user root. Aug 15 06:08:15 oleg133-client sshd[20256]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:08:15 oleg133-client sshd[20269]: Received disconnect from 192.168.201.33 port 51696:11: disconnected by user Aug 15 06:08:15 oleg133-client sshd[20269]: Disconnected from user root 192.168.201.33 port 51696 Aug 15 06:08:15 oleg133-client sshd[20256]: pam_unix(sshd:session): session closed for user root Aug 15 06:08:16 oleg133-client systemd[1]: session-85.scope: Succeeded. Aug 15 06:08:16 oleg133-client systemd-logind[1034]: Session 85 logged out. Waiting for processes to exit. Aug 15 06:08:16 oleg133-client systemd-logind[1034]: Removed session 85. Aug 15 06:08:17 oleg133-client sshd[20393]: Accepted publickey for root from 192.168.201.33 port 53202 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:08:17 oleg133-client systemd[1]: Started Session 86 of user root. Aug 15 06:08:17 oleg133-client systemd-logind[1034]: New session 86 of user root. Aug 15 06:08:17 oleg133-client sshd[20393]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:08:17 oleg133-client sshd[20406]: Received disconnect from 192.168.201.33 port 53202:11: disconnected by user Aug 15 06:08:17 oleg133-client sshd[20406]: Disconnected from user root 192.168.201.33 port 53202 Aug 15 06:08:17 oleg133-client sshd[20393]: pam_unix(sshd:session): session closed for user root Aug 15 06:08:18 oleg133-client systemd[1]: session-86.scope: Succeeded. Aug 15 06:08:18 oleg133-client systemd-logind[1034]: Session 86 logged out. Waiting for processes to exit. Aug 15 06:08:18 oleg133-client systemd-logind[1034]: Removed session 86. Aug 15 06:08:19 oleg133-client sshd[20537]: Accepted publickey for root from 192.168.201.33 port 53210 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:08:19 oleg133-client systemd-logind[1034]: New session 87 of user root. Aug 15 06:08:19 oleg133-client systemd[1]: Started Session 87 of user root. Aug 15 06:08:19 oleg133-client sshd[20537]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:08:19 oleg133-client sshd[20550]: Received disconnect from 192.168.201.33 port 53210:11: disconnected by user Aug 15 06:08:19 oleg133-client sshd[20550]: Disconnected from user root 192.168.201.33 port 53210 Aug 15 06:08:19 oleg133-client sshd[20537]: pam_unix(sshd:session): session closed for user root Aug 15 06:08:20 oleg133-client systemd-logind[1034]: Session 87 logged out. Waiting for processes to exit. Aug 15 06:08:20 oleg133-client systemd[1]: session-87.scope: Succeeded. Aug 15 06:08:20 oleg133-client systemd-logind[1034]: Removed session 87. Aug 15 06:08:21 oleg133-client sshd[20691]: Accepted publickey for root from 192.168.201.33 port 53222 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:08:21 oleg133-client systemd-logind[1034]: New session 88 of user root. Aug 15 06:08:21 oleg133-client systemd[1]: Started Session 88 of user root. Aug 15 06:08:21 oleg133-client sshd[20691]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:08:22 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 17a: timeout bulk get, don't evict client (2732) ========================================================== 06:08:20 (1786788500) Aug 15 06:08:22 oleg133-client sshd[20704]: Received disconnect from 192.168.201.33 port 53222:11: disconnected by user Aug 15 06:08:22 oleg133-client sshd[20704]: Disconnected from user root 192.168.201.33 port 53222 Aug 15 06:08:22 oleg133-client sshd[20691]: pam_unix(sshd:session): session closed for user root Aug 15 06:08:22 oleg133-client systemd-logind[1034]: Session 88 logged out. Waiting for processes to exit. Aug 15 06:08:22 oleg133-client systemd[1]: session-88.scope: Succeeded. Aug 15 06:08:22 oleg133-client systemd-logind[1034]: Removed session 88. Aug 15 06:09:10 oleg133-client sshd[21044]: Accepted publickey for root from 192.168.201.33 port 59478 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:10 oleg133-client systemd-logind[1034]: New session 89 of user root. Aug 15 06:09:10 oleg133-client systemd[1]: Started Session 89 of user root. Aug 15 06:09:10 oleg133-client sshd[21044]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:11 oleg133-client sshd[21057]: Received disconnect from 192.168.201.33 port 59478:11: disconnected by user Aug 15 06:09:11 oleg133-client sshd[21057]: Disconnected from user root 192.168.201.33 port 59478 Aug 15 06:09:11 oleg133-client sshd[21044]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:11 oleg133-client systemd[1]: session-89.scope: Succeeded. Aug 15 06:09:11 oleg133-client systemd-logind[1034]: Session 89 logged out. Waiting for processes to exit. Aug 15 06:09:11 oleg133-client systemd-logind[1034]: Removed session 89. Aug 15 06:09:12 oleg133-client sshd[21181]: Accepted publickey for root from 192.168.201.33 port 59486 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:12 oleg133-client systemd[1]: Started Session 90 of user root. Aug 15 06:09:12 oleg133-client systemd-logind[1034]: New session 90 of user root. Aug 15 06:09:12 oleg133-client sshd[21181]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:12 oleg133-client sshd[21194]: Received disconnect from 192.168.201.33 port 59486:11: disconnected by user Aug 15 06:09:12 oleg133-client sshd[21194]: Disconnected from user root 192.168.201.33 port 59486 Aug 15 06:09:12 oleg133-client sshd[21181]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:12 oleg133-client systemd[1]: session-90.scope: Succeeded. Aug 15 06:09:12 oleg133-client systemd-logind[1034]: Session 90 logged out. Waiting for processes to exit. Aug 15 06:09:12 oleg133-client systemd-logind[1034]: Removed session 90. Aug 15 06:09:13 oleg133-client sshd[21325]: Accepted publickey for root from 192.168.201.33 port 59502 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:13 oleg133-client systemd-logind[1034]: New session 91 of user root. Aug 15 06:09:13 oleg133-client systemd[1]: Started Session 91 of user root. Aug 15 06:09:13 oleg133-client sshd[21325]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:13 oleg133-client sshd[21338]: Received disconnect from 192.168.201.33 port 59502:11: disconnected by user Aug 15 06:09:13 oleg133-client sshd[21338]: Disconnected from user root 192.168.201.33 port 59502 Aug 15 06:09:13 oleg133-client sshd[21325]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:14 oleg133-client systemd[1]: session-91.scope: Succeeded. Aug 15 06:09:14 oleg133-client systemd-logind[1034]: Session 91 logged out. Waiting for processes to exit. Aug 15 06:09:14 oleg133-client systemd-logind[1034]: Removed session 91. Aug 15 06:09:14 oleg133-client sshd[21478]: Accepted publickey for root from 192.168.201.33 port 59508 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:14 oleg133-client systemd[1]: Started Session 92 of user root. Aug 15 06:09:14 oleg133-client systemd-logind[1034]: New session 92 of user root. Aug 15 06:09:14 oleg133-client sshd[21478]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:15 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 17b: timeout bulk get, dont evict client (3582) ========================================================== 06:09:14 (1786788554) Aug 15 06:09:15 oleg133-client sshd[21491]: Received disconnect from 192.168.201.33 port 59508:11: disconnected by user Aug 15 06:09:15 oleg133-client sshd[21491]: Disconnected from user root 192.168.201.33 port 59508 Aug 15 06:09:15 oleg133-client sshd[21478]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:15 oleg133-client systemd[1]: session-92.scope: Succeeded. Aug 15 06:09:15 oleg133-client systemd-logind[1034]: Session 92 logged out. Waiting for processes to exit. Aug 15 06:09:15 oleg133-client systemd-logind[1034]: Removed session 92. Aug 15 06:09:16 oleg133-client sshd[21614]: Accepted publickey for root from 192.168.201.33 port 43792 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:16 oleg133-client systemd-logind[1034]: New session 93 of user root. Aug 15 06:09:16 oleg133-client systemd[1]: Started Session 93 of user root. Aug 15 06:09:16 oleg133-client sshd[21614]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:16 oleg133-client kernel: Lustre: DEBUG MARKER: SKIP: recovery-small test_17b Needs multiple clients Aug 15 06:09:16 oleg133-client sshd[21627]: Received disconnect from 192.168.201.33 port 43792:11: disconnected by user Aug 15 06:09:16 oleg133-client sshd[21627]: Disconnected from user root 192.168.201.33 port 43792 Aug 15 06:09:16 oleg133-client sshd[21614]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:16 oleg133-client systemd-logind[1034]: Session 93 logged out. Waiting for processes to exit. Aug 15 06:09:16 oleg133-client systemd[1]: session-93.scope: Succeeded. Aug 15 06:09:16 oleg133-client systemd-logind[1034]: Removed session 93. Aug 15 06:09:18 oleg133-client sshd[21767]: Accepted publickey for root from 192.168.201.33 port 43794 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:18 oleg133-client systemd-logind[1034]: New session 94 of user root. Aug 15 06:09:18 oleg133-client systemd[1]: Started Session 94 of user root. Aug 15 06:09:18 oleg133-client sshd[21767]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:18 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 18a: manual ost invalidate clears page cache immediately ========================================================== 06:09:16 (1786788556) Aug 15 06:09:18 oleg133-client sshd[21780]: Received disconnect from 192.168.201.33 port 43794:11: disconnected by user Aug 15 06:09:18 oleg133-client sshd[21780]: Disconnected from user root 192.168.201.33 port 43794 Aug 15 06:09:18 oleg133-client sshd[21767]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:18 oleg133-client systemd[1]: session-94.scope: Succeeded. Aug 15 06:09:18 oleg133-client systemd-logind[1034]: Session 94 logged out. Waiting for processes to exit. Aug 15 06:09:18 oleg133-client systemd-logind[1034]: Removed session 94. Aug 15 06:09:19 oleg133-client kernel: Lustre: setting import lustre-OST0001_UUID INACTIVE by administrator request Aug 15 06:09:19 oleg133-client kernel: Lustre: lustre-OST0001-osc-ffff993744fd2000: Connection to lustre-OST0001 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 06:09:19 oleg133-client kernel: Lustre: Skipped 5 previous similar messages Aug 15 06:09:19 oleg133-client kernel: LustreError: lustre-OST0001-osc-ffff993744fd2000: This client was evicted by lustre-OST0001; in progress operations using this service will fail. Aug 15 06:09:19 oleg133-client kernel: Lustre: lustre-OST0001-osc-ffff993744fd2000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 06:09:19 oleg133-client kernel: Lustre: Skipped 4 previous similar messages Aug 15 06:09:20 oleg133-client sshd[21943]: Accepted publickey for root from 192.168.201.33 port 43798 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:20 oleg133-client systemd[1]: Started Session 95 of user root. Aug 15 06:09:20 oleg133-client systemd-logind[1034]: New session 95 of user root. Aug 15 06:09:20 oleg133-client sshd[21943]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:20 oleg133-client sshd[21956]: Received disconnect from 192.168.201.33 port 43798:11: disconnected by user Aug 15 06:09:20 oleg133-client sshd[21956]: Disconnected from user root 192.168.201.33 port 43798 Aug 15 06:09:20 oleg133-client sshd[21943]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:21 oleg133-client systemd[1]: session-95.scope: Succeeded. Aug 15 06:09:21 oleg133-client systemd-logind[1034]: Session 95 logged out. Waiting for processes to exit. Aug 15 06:09:21 oleg133-client systemd-logind[1034]: Removed session 95. Aug 15 06:09:22 oleg133-client sshd[22080]: Accepted publickey for root from 192.168.201.33 port 43804 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:22 oleg133-client systemd-logind[1034]: New session 96 of user root. Aug 15 06:09:22 oleg133-client systemd[1]: Started Session 96 of user root. Aug 15 06:09:22 oleg133-client sshd[22080]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:22 oleg133-client sshd[22093]: Received disconnect from 192.168.201.33 port 43804:11: disconnected by user Aug 15 06:09:22 oleg133-client sshd[22093]: Disconnected from user root 192.168.201.33 port 43804 Aug 15 06:09:22 oleg133-client sshd[22080]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:22 oleg133-client systemd[1]: session-96.scope: Succeeded. Aug 15 06:09:22 oleg133-client systemd-logind[1034]: Session 96 logged out. Waiting for processes to exit. Aug 15 06:09:22 oleg133-client systemd-logind[1034]: Removed session 96. Aug 15 06:09:24 oleg133-client sshd[22225]: Accepted publickey for root from 192.168.201.33 port 43816 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:24 oleg133-client systemd-logind[1034]: New session 97 of user root. Aug 15 06:09:24 oleg133-client systemd[1]: Started Session 97 of user root. Aug 15 06:09:24 oleg133-client sshd[22225]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:24 oleg133-client sshd[22238]: Received disconnect from 192.168.201.33 port 43816:11: disconnected by user Aug 15 06:09:24 oleg133-client sshd[22238]: Disconnected from user root 192.168.201.33 port 43816 Aug 15 06:09:24 oleg133-client sshd[22225]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:24 oleg133-client systemd[1]: session-97.scope: Succeeded. Aug 15 06:09:24 oleg133-client systemd-logind[1034]: Session 97 logged out. Waiting for processes to exit. Aug 15 06:09:24 oleg133-client systemd-logind[1034]: Removed session 97. Aug 15 06:09:26 oleg133-client sshd[22378]: Accepted publickey for root from 192.168.201.33 port 37994 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:26 oleg133-client systemd[1]: Started Session 98 of user root. Aug 15 06:09:26 oleg133-client systemd-logind[1034]: New session 98 of user root. Aug 15 06:09:26 oleg133-client sshd[22378]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:26 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 18b: eviction and reconnect clears page cache (2766) ========================================================== 06:09:25 (1786788565) Aug 15 06:09:26 oleg133-client sshd[22391]: Received disconnect from 192.168.201.33 port 37994:11: disconnected by user Aug 15 06:09:26 oleg133-client sshd[22391]: Disconnected from user root 192.168.201.33 port 37994 Aug 15 06:09:26 oleg133-client sshd[22378]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:27 oleg133-client systemd[1]: session-98.scope: Succeeded. Aug 15 06:09:27 oleg133-client systemd-logind[1034]: Session 98 logged out. Waiting for processes to exit. Aug 15 06:09:27 oleg133-client systemd-logind[1034]: Removed session 98. Aug 15 06:09:33 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. Aug 15 06:09:52 oleg133-client sshd[22594]: Accepted publickey for root from 192.168.201.33 port 42026 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:52 oleg133-client systemd-logind[1034]: New session 99 of user root. Aug 15 06:09:52 oleg133-client systemd[1]: Started Session 99 of user root. Aug 15 06:09:52 oleg133-client sshd[22594]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:52 oleg133-client sshd[22607]: Received disconnect from 192.168.201.33 port 42026:11: disconnected by user Aug 15 06:09:52 oleg133-client sshd[22607]: Disconnected from user root 192.168.201.33 port 42026 Aug 15 06:09:52 oleg133-client sshd[22594]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:52 oleg133-client systemd[1]: session-99.scope: Succeeded. Aug 15 06:09:52 oleg133-client systemd-logind[1034]: Session 99 logged out. Waiting for processes to exit. Aug 15 06:09:52 oleg133-client systemd-logind[1034]: Removed session 99. Aug 15 06:09:54 oleg133-client sshd[22731]: Accepted publickey for root from 192.168.201.33 port 42036 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:54 oleg133-client systemd-logind[1034]: New session 100 of user root. Aug 15 06:09:54 oleg133-client systemd[1]: Started Session 100 of user root. Aug 15 06:09:54 oleg133-client sshd[22731]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:54 oleg133-client sshd[22744]: Received disconnect from 192.168.201.33 port 42036:11: disconnected by user Aug 15 06:09:54 oleg133-client sshd[22744]: Disconnected from user root 192.168.201.33 port 42036 Aug 15 06:09:54 oleg133-client sshd[22731]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:54 oleg133-client systemd[1]: session-100.scope: Succeeded. Aug 15 06:09:54 oleg133-client systemd-logind[1034]: Session 100 logged out. Waiting for processes to exit. Aug 15 06:09:54 oleg133-client systemd-logind[1034]: Removed session 100. Aug 15 06:09:55 oleg133-client sshd[22875]: Accepted publickey for root from 192.168.201.33 port 52826 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:55 oleg133-client systemd[1]: Started Session 101 of user root. Aug 15 06:09:55 oleg133-client systemd-logind[1034]: New session 101 of user root. Aug 15 06:09:55 oleg133-client sshd[22875]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:56 oleg133-client sshd[22888]: Received disconnect from 192.168.201.33 port 52826:11: disconnected by user Aug 15 06:09:56 oleg133-client sshd[22888]: Disconnected from user root 192.168.201.33 port 52826 Aug 15 06:09:56 oleg133-client sshd[22875]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:56 oleg133-client systemd[1]: session-101.scope: Succeeded. Aug 15 06:09:56 oleg133-client systemd-logind[1034]: Session 101 logged out. Waiting for processes to exit. Aug 15 06:09:56 oleg133-client systemd-logind[1034]: Removed session 101. Aug 15 06:09:57 oleg133-client sshd[23029]: Accepted publickey for root from 192.168.201.33 port 52842 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:09:57 oleg133-client systemd[1]: Started Session 102 of user root. Aug 15 06:09:57 oleg133-client systemd-logind[1034]: New session 102 of user root. Aug 15 06:09:57 oleg133-client sshd[23029]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:09:57 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 18c: Dropped connect reply after eviction handing (14755) ========================================================== 06:09:56 (1786788596) Aug 15 06:09:57 oleg133-client sshd[23042]: Received disconnect from 192.168.201.33 port 52842:11: disconnected by user Aug 15 06:09:57 oleg133-client sshd[23042]: Disconnected from user root 192.168.201.33 port 52842 Aug 15 06:09:57 oleg133-client sshd[23029]: pam_unix(sshd:session): session closed for user root Aug 15 06:09:57 oleg133-client systemd[1]: session-102.scope: Succeeded. Aug 15 06:09:57 oleg133-client systemd-logind[1034]: Session 102 logged out. Waiting for processes to exit. Aug 15 06:09:57 oleg133-client systemd-logind[1034]: Removed session 102. Aug 15 06:09:59 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. Aug 15 06:10:14 oleg133-client sshd[23254]: Accepted publickey for root from 192.168.201.33 port 34894 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:10:14 oleg133-client systemd[1]: Started Session 103 of user root. Aug 15 06:10:14 oleg133-client systemd-logind[1034]: New session 103 of user root. Aug 15 06:10:14 oleg133-client sshd[23254]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:10:15 oleg133-client sshd[23267]: Received disconnect from 192.168.201.33 port 34894:11: disconnected by user Aug 15 06:10:15 oleg133-client sshd[23267]: Disconnected from user root 192.168.201.33 port 34894 Aug 15 06:10:15 oleg133-client sshd[23254]: pam_unix(sshd:session): session closed for user root Aug 15 06:10:15 oleg133-client systemd[1]: session-103.scope: Succeeded. Aug 15 06:10:15 oleg133-client systemd-logind[1034]: Session 103 logged out. Waiting for processes to exit. Aug 15 06:10:15 oleg133-client systemd-logind[1034]: Removed session 103. Aug 15 06:10:17 oleg133-client sshd[23391]: Accepted publickey for root from 192.168.201.33 port 57844 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:10:17 oleg133-client systemd-logind[1034]: New session 104 of user root. Aug 15 06:10:17 oleg133-client systemd[1]: Started Session 104 of user root. Aug 15 06:10:17 oleg133-client sshd[23391]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:10:17 oleg133-client sshd[23404]: Received disconnect from 192.168.201.33 port 57844:11: disconnected by user Aug 15 06:10:17 oleg133-client sshd[23404]: Disconnected from user root 192.168.201.33 port 57844 Aug 15 06:10:17 oleg133-client sshd[23391]: pam_unix(sshd:session): session closed for user root Aug 15 06:10:17 oleg133-client systemd[1]: session-104.scope: Succeeded. Aug 15 06:10:17 oleg133-client systemd-logind[1034]: Session 104 logged out. Waiting for processes to exit. Aug 15 06:10:17 oleg133-client systemd-logind[1034]: Removed session 104. Aug 15 06:10:18 oleg133-client sshd[23536]: Accepted publickey for root from 192.168.201.33 port 57858 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:10:18 oleg133-client systemd-logind[1034]: New session 105 of user root. Aug 15 06:10:18 oleg133-client systemd[1]: Started Session 105 of user root. Aug 15 06:10:18 oleg133-client sshd[23536]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:10:18 oleg133-client sshd[23549]: Received disconnect from 192.168.201.33 port 57858:11: disconnected by user Aug 15 06:10:19 oleg133-client sshd[23549]: Disconnected from user root 192.168.201.33 port 57858 Aug 15 06:10:19 oleg133-client sshd[23536]: pam_unix(sshd:session): session closed for user root Aug 15 06:10:19 oleg133-client systemd-logind[1034]: Session 105 logged out. Waiting for processes to exit. Aug 15 06:10:19 oleg133-client systemd[1]: session-105.scope: Succeeded. Aug 15 06:10:19 oleg133-client systemd-logind[1034]: Removed session 105. Aug 15 06:10:20 oleg133-client sshd[23689]: Accepted publickey for root from 192.168.201.33 port 57868 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:10:20 oleg133-client systemd-logind[1034]: New session 106 of user root. Aug 15 06:10:20 oleg133-client systemd[1]: Started Session 106 of user root. Aug 15 06:10:20 oleg133-client sshd[23689]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:10:20 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 19a: test expired_lock_main on mds (2867) ========================================================== 06:10:19 (1786788619) Aug 15 06:10:20 oleg133-client sshd[23702]: Received disconnect from 192.168.201.33 port 57868:11: disconnected by user Aug 15 06:10:20 oleg133-client sshd[23702]: Disconnected from user root 192.168.201.33 port 57868 Aug 15 06:10:20 oleg133-client sshd[23689]: pam_unix(sshd:session): session closed for user root Aug 15 06:10:20 oleg133-client systemd-logind[1034]: Session 106 logged out. Waiting for processes to exit. Aug 15 06:10:20 oleg133-client systemd[1]: session-106.scope: Succeeded. Aug 15 06:10:21 oleg133-client systemd-logind[1034]: Removed session 106. Aug 15 06:10:21 oleg133-client systemd-udevd[23756]: Process '/home/green/git/lustre-release/lustre/utils/lctl set_param 'lod.*.mdt_hash=crush'' failed with exit code 2. Aug 15 06:10:21 oleg133-client kernel: Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 Aug 15 06:10:21 oleg133-client kernel: Lustre: Skipped 1 previous similar message Aug 15 06:10:54 oleg133-client kernel: Lustre: 2376:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786788638/real 1786788638] req@ffff993760047480 x1873582575751424/t0(0) o103->lustre-MDT0000-mdc-ffff993744fd2000@192.168.201.133@tcp:17/18 lens 328/224 e 0 to 1 dl 1786788654 ref 1 fl Rpc:XQr/202/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 Aug 15 06:10:54 oleg133-client kernel: Lustre: 2376:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Aug 15 06:12:04 oleg133-client kernel: LustreError: lustre-MDT0000-mdc-ffff993744fd2000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. Aug 15 06:12:06 oleg133-client systemd[1]: mnt-lustre2.mount: Succeeded. Aug 15 06:12:06 oleg133-client kernel: Lustre: Unmounted lustre-client Aug 15 06:12:09 oleg133-client sshd[24042]: Accepted publickey for root from 192.168.201.33 port 48896 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:12:09 oleg133-client systemd-logind[1034]: New session 107 of user root. Aug 15 06:12:09 oleg133-client systemd[1]: Started Session 107 of user root. Aug 15 06:12:09 oleg133-client sshd[24042]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:12:09 oleg133-client sshd[24055]: Received disconnect from 192.168.201.33 port 48896:11: disconnected by user Aug 15 06:12:09 oleg133-client sshd[24055]: Disconnected from user root 192.168.201.33 port 48896 Aug 15 06:12:09 oleg133-client sshd[24042]: pam_unix(sshd:session): session closed for user root Aug 15 06:12:09 oleg133-client systemd[1]: session-107.scope: Succeeded. Aug 15 06:12:09 oleg133-client systemd-logind[1034]: Session 107 logged out. Waiting for processes to exit. Aug 15 06:12:09 oleg133-client systemd-logind[1034]: Removed session 107. Aug 15 06:12:10 oleg133-client sshd[24180]: Accepted publickey for root from 192.168.201.33 port 48900 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:12:10 oleg133-client systemd-logind[1034]: New session 108 of user root. Aug 15 06:12:10 oleg133-client systemd[1]: Started Session 108 of user root. Aug 15 06:12:10 oleg133-client sshd[24180]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:12:11 oleg133-client sshd[24193]: Received disconnect from 192.168.201.33 port 48900:11: disconnected by user Aug 15 06:12:11 oleg133-client sshd[24193]: Disconnected from user root 192.168.201.33 port 48900 Aug 15 06:12:11 oleg133-client sshd[24180]: pam_unix(sshd:session): session closed for user root Aug 15 06:12:11 oleg133-client systemd[1]: session-108.scope: Succeeded. Aug 15 06:12:11 oleg133-client systemd-logind[1034]: Session 108 logged out. Waiting for processes to exit. Aug 15 06:12:11 oleg133-client systemd-logind[1034]: Removed session 108. Aug 15 06:12:12 oleg133-client sshd[24324]: Accepted publickey for root from 192.168.201.33 port 48904 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:12:13 oleg133-client systemd-logind[1034]: New session 109 of user root. Aug 15 06:12:13 oleg133-client systemd[1]: Started Session 109 of user root. Aug 15 06:12:13 oleg133-client sshd[24324]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:12:13 oleg133-client sshd[24338]: Received disconnect from 192.168.201.33 port 48904:11: disconnected by user Aug 15 06:12:13 oleg133-client sshd[24338]: Disconnected from user root 192.168.201.33 port 48904 Aug 15 06:12:13 oleg133-client sshd[24324]: pam_unix(sshd:session): session closed for user root Aug 15 06:12:13 oleg133-client systemd-logind[1034]: Session 109 logged out. Waiting for processes to exit. Aug 15 06:12:13 oleg133-client systemd[1]: session-109.scope: Succeeded. Aug 15 06:12:13 oleg133-client systemd-logind[1034]: Removed session 109. Aug 15 06:12:14 oleg133-client sshd[24478]: Accepted publickey for root from 192.168.201.33 port 48908 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:12:14 oleg133-client systemd-logind[1034]: New session 110 of user root. Aug 15 06:12:14 oleg133-client systemd[1]: Started Session 110 of user root. Aug 15 06:12:14 oleg133-client sshd[24478]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:12:15 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 19b: test expired_lock_main on ost (2867) ========================================================== 06:12:14 (1786788734) Aug 15 06:12:15 oleg133-client sshd[24491]: Received disconnect from 192.168.201.33 port 48908:11: disconnected by user Aug 15 06:12:15 oleg133-client sshd[24491]: Disconnected from user root 192.168.201.33 port 48908 Aug 15 06:12:15 oleg133-client sshd[24478]: pam_unix(sshd:session): session closed for user root Aug 15 06:12:15 oleg133-client systemd-logind[1034]: Session 110 logged out. Waiting for processes to exit. Aug 15 06:12:15 oleg133-client systemd[1]: session-110.scope: Succeeded. Aug 15 06:12:15 oleg133-client systemd-logind[1034]: Removed session 110. Aug 15 06:12:15 oleg133-client systemd-udevd[24546]: Process '/home/green/git/lustre-release/lustre/utils/lctl set_param 'lod.*.mdt_hash=crush'' failed with exit code 2. Aug 15 06:12:15 oleg133-client kernel: Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 Aug 15 06:14:00 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. Aug 15 06:14:02 oleg133-client systemd[1]: mnt-lustre2.mount: Succeeded. Aug 15 06:14:03 oleg133-client kernel: Lustre: Unmounted lustre-client Aug 15 06:14:05 oleg133-client sshd[24829]: Accepted publickey for root from 192.168.201.33 port 57614 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:05 oleg133-client systemd-logind[1034]: New session 111 of user root. Aug 15 06:14:05 oleg133-client systemd[1]: Started Session 111 of user root. Aug 15 06:14:05 oleg133-client sshd[24829]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:05 oleg133-client sshd[24842]: Received disconnect from 192.168.201.33 port 57614:11: disconnected by user Aug 15 06:14:05 oleg133-client sshd[24842]: Disconnected from user root 192.168.201.33 port 57614 Aug 15 06:14:05 oleg133-client sshd[24829]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:05 oleg133-client systemd[1]: session-111.scope: Succeeded. Aug 15 06:14:05 oleg133-client systemd-logind[1034]: Session 111 logged out. Waiting for processes to exit. Aug 15 06:14:05 oleg133-client systemd-logind[1034]: Removed session 111. Aug 15 06:14:07 oleg133-client sshd[24967]: Accepted publickey for root from 192.168.201.33 port 35788 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:07 oleg133-client systemd-logind[1034]: New session 112 of user root. Aug 15 06:14:07 oleg133-client sshd[24967]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:07 oleg133-client systemd[1]: Started Session 112 of user root. Aug 15 06:14:07 oleg133-client sshd[24980]: Received disconnect from 192.168.201.33 port 35788:11: disconnected by user Aug 15 06:14:07 oleg133-client sshd[24980]: Disconnected from user root 192.168.201.33 port 35788 Aug 15 06:14:07 oleg133-client sshd[24967]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:08 oleg133-client systemd-logind[1034]: Session 112 logged out. Waiting for processes to exit. Aug 15 06:14:08 oleg133-client systemd[1]: session-112.scope: Succeeded. Aug 15 06:14:08 oleg133-client systemd-logind[1034]: Removed session 112. Aug 15 06:14:09 oleg133-client sshd[25112]: Accepted publickey for root from 192.168.201.33 port 35796 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:09 oleg133-client systemd[1]: Started Session 113 of user root. Aug 15 06:14:09 oleg133-client systemd-logind[1034]: New session 113 of user root. Aug 15 06:14:09 oleg133-client sshd[25112]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:09 oleg133-client sshd[25125]: Received disconnect from 192.168.201.33 port 35796:11: disconnected by user Aug 15 06:14:09 oleg133-client sshd[25125]: Disconnected from user root 192.168.201.33 port 35796 Aug 15 06:14:10 oleg133-client sshd[25112]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:10 oleg133-client systemd[1]: session-113.scope: Succeeded. Aug 15 06:14:10 oleg133-client systemd-logind[1034]: Session 113 logged out. Waiting for processes to exit. Aug 15 06:14:10 oleg133-client systemd-logind[1034]: Removed session 113. Aug 15 06:14:12 oleg133-client sshd[25265]: Accepted publickey for root from 192.168.201.33 port 35810 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:12 oleg133-client systemd-logind[1034]: New session 114 of user root. Aug 15 06:14:12 oleg133-client systemd[1]: Started Session 114 of user root. Aug 15 06:14:12 oleg133-client sshd[25265]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:12 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 19c: check reconnect and lock resend do not trigger expired_lock_main ========================================================== 06:14:10 (1786788850) Aug 15 06:14:12 oleg133-client sshd[25278]: Received disconnect from 192.168.201.33 port 35810:11: disconnected by user Aug 15 06:14:12 oleg133-client sshd[25278]: Disconnected from user root 192.168.201.33 port 35810 Aug 15 06:14:12 oleg133-client sshd[25265]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:12 oleg133-client systemd[1]: session-114.scope: Succeeded. Aug 15 06:14:12 oleg133-client systemd-logind[1034]: Session 114 logged out. Waiting for processes to exit. Aug 15 06:14:12 oleg133-client systemd-logind[1034]: Removed session 114. Aug 15 06:14:13 oleg133-client systemd-udevd[25332]: Process '/home/green/git/lustre-release/lustre/utils/lctl set_param 'lod.*.mdt_hash=crush'' failed with exit code 2. Aug 15 06:14:13 oleg133-client kernel: Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 Aug 15 06:14:20 oleg133-client systemd[1]: mnt-lustre2.mount: Succeeded. Aug 15 06:14:20 oleg133-client kernel: Lustre: Unmounted lustre-client Aug 15 06:14:26 oleg133-client sshd[25477]: Accepted publickey for root from 192.168.201.33 port 60900 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:26 oleg133-client systemd[1]: Started Session 115 of user root. Aug 15 06:14:26 oleg133-client systemd-logind[1034]: New session 115 of user root. Aug 15 06:14:26 oleg133-client sshd[25477]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:26 oleg133-client sshd[25490]: Received disconnect from 192.168.201.33 port 60900:11: disconnected by user Aug 15 06:14:27 oleg133-client sshd[25490]: Disconnected from user root 192.168.201.33 port 60900 Aug 15 06:14:27 oleg133-client sshd[25477]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:27 oleg133-client systemd-logind[1034]: Session 115 logged out. Waiting for processes to exit. Aug 15 06:14:27 oleg133-client systemd[1]: session-115.scope: Succeeded. Aug 15 06:14:27 oleg133-client systemd-logind[1034]: Removed session 115. Aug 15 06:14:29 oleg133-client sshd[25614]: Accepted publickey for root from 192.168.201.33 port 60910 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:29 oleg133-client systemd[1]: Started Session 116 of user root. Aug 15 06:14:29 oleg133-client systemd-logind[1034]: New session 116 of user root. Aug 15 06:14:29 oleg133-client sshd[25614]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:29 oleg133-client sshd[25627]: Received disconnect from 192.168.201.33 port 60910:11: disconnected by user Aug 15 06:14:29 oleg133-client sshd[25627]: Disconnected from user root 192.168.201.33 port 60910 Aug 15 06:14:29 oleg133-client sshd[25614]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:29 oleg133-client systemd-logind[1034]: Session 116 logged out. Waiting for processes to exit. Aug 15 06:14:29 oleg133-client systemd[1]: session-116.scope: Succeeded. Aug 15 06:14:29 oleg133-client systemd-logind[1034]: Removed session 116. Aug 15 06:14:31 oleg133-client sshd[25759]: Accepted publickey for root from 192.168.201.33 port 60916 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:31 oleg133-client systemd[1]: Started Session 117 of user root. Aug 15 06:14:31 oleg133-client systemd-logind[1034]: New session 117 of user root. Aug 15 06:14:31 oleg133-client sshd[25759]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:31 oleg133-client sshd[25772]: Received disconnect from 192.168.201.33 port 60916:11: disconnected by user Aug 15 06:14:31 oleg133-client sshd[25772]: Disconnected from user root 192.168.201.33 port 60916 Aug 15 06:14:31 oleg133-client sshd[25759]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:32 oleg133-client systemd[1]: session-117.scope: Succeeded. Aug 15 06:14:32 oleg133-client systemd-logind[1034]: Session 117 logged out. Waiting for processes to exit. Aug 15 06:14:32 oleg133-client systemd-logind[1034]: Removed session 117. Aug 15 06:14:33 oleg133-client sshd[25913]: Accepted publickey for root from 192.168.201.33 port 60928 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:33 oleg133-client systemd-logind[1034]: New session 118 of user root. Aug 15 06:14:33 oleg133-client systemd[1]: Started Session 118 of user root. Aug 15 06:14:33 oleg133-client sshd[25913]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:33 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 20a: ldlm_handle_enqueue error (should return error) ========================================================== 06:14:32 (1786788872) Aug 15 06:14:33 oleg133-client sshd[25926]: Received disconnect from 192.168.201.33 port 60928:11: disconnected by user Aug 15 06:14:34 oleg133-client sshd[25926]: Disconnected from user root 192.168.201.33 port 60928 Aug 15 06:14:34 oleg133-client sshd[25913]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:34 oleg133-client systemd[1]: session-118.scope: Succeeded. Aug 15 06:14:34 oleg133-client systemd-logind[1034]: Session 118 logged out. Waiting for processes to exit. Aug 15 06:14:34 oleg133-client systemd-logind[1034]: Removed session 118. Aug 15 06:14:35 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: operation ldlm_enqueue to node 192.168.201.133@tcp failed: rc = -12 Aug 15 06:14:35 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:14:36 oleg133-client sshd[26095]: Accepted publickey for root from 192.168.201.33 port 50740 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:36 oleg133-client systemd-logind[1034]: New session 119 of user root. Aug 15 06:14:36 oleg133-client systemd[1]: Started Session 119 of user root. Aug 15 06:14:36 oleg133-client sshd[26095]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:36 oleg133-client sshd[26108]: Received disconnect from 192.168.201.33 port 50740:11: disconnected by user Aug 15 06:14:36 oleg133-client sshd[26108]: Disconnected from user root 192.168.201.33 port 50740 Aug 15 06:14:36 oleg133-client sshd[26095]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:36 oleg133-client systemd[1]: session-119.scope: Succeeded. Aug 15 06:14:36 oleg133-client systemd-logind[1034]: Session 119 logged out. Waiting for processes to exit. Aug 15 06:14:36 oleg133-client systemd-logind[1034]: Removed session 119. Aug 15 06:14:37 oleg133-client sshd[26232]: Accepted publickey for root from 192.168.201.33 port 50748 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:38 oleg133-client systemd-logind[1034]: New session 120 of user root. Aug 15 06:14:38 oleg133-client systemd[1]: Started Session 120 of user root. Aug 15 06:14:38 oleg133-client sshd[26232]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:38 oleg133-client sshd[26245]: Received disconnect from 192.168.201.33 port 50748:11: disconnected by user Aug 15 06:14:38 oleg133-client sshd[26245]: Disconnected from user root 192.168.201.33 port 50748 Aug 15 06:14:38 oleg133-client sshd[26232]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:38 oleg133-client systemd[1]: session-120.scope: Succeeded. Aug 15 06:14:38 oleg133-client systemd-logind[1034]: Session 120 logged out. Waiting for processes to exit. Aug 15 06:14:38 oleg133-client systemd-logind[1034]: Removed session 120. Aug 15 06:14:39 oleg133-client sshd[26377]: Accepted publickey for root from 192.168.201.33 port 50760 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:39 oleg133-client systemd-logind[1034]: New session 121 of user root. Aug 15 06:14:39 oleg133-client systemd[1]: Started Session 121 of user root. Aug 15 06:14:39 oleg133-client sshd[26377]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:40 oleg133-client sshd[26390]: Received disconnect from 192.168.201.33 port 50760:11: disconnected by user Aug 15 06:14:40 oleg133-client sshd[26390]: Disconnected from user root 192.168.201.33 port 50760 Aug 15 06:14:40 oleg133-client sshd[26377]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:40 oleg133-client systemd[1]: session-121.scope: Succeeded. Aug 15 06:14:40 oleg133-client systemd-logind[1034]: Session 121 logged out. Waiting for processes to exit. Aug 15 06:14:40 oleg133-client systemd-logind[1034]: Removed session 121. Aug 15 06:14:41 oleg133-client sshd[26530]: Accepted publickey for root from 192.168.201.33 port 50762 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:41 oleg133-client systemd[1]: Started Session 122 of user root. Aug 15 06:14:41 oleg133-client systemd-logind[1034]: New session 122 of user root. Aug 15 06:14:41 oleg133-client sshd[26530]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:42 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 20b: ldlm_handle_enqueue error (should return error) ========================================================== 06:14:40 (1786788880) Aug 15 06:14:42 oleg133-client sshd[26543]: Received disconnect from 192.168.201.33 port 50762:11: disconnected by user Aug 15 06:14:42 oleg133-client sshd[26543]: Disconnected from user root 192.168.201.33 port 50762 Aug 15 06:14:42 oleg133-client sshd[26530]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:42 oleg133-client systemd-logind[1034]: Session 122 logged out. Waiting for processes to exit. Aug 15 06:14:42 oleg133-client systemd[1]: session-122.scope: Succeeded. Aug 15 06:14:42 oleg133-client systemd-logind[1034]: Removed session 122. Aug 15 06:14:43 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: operation ldlm_enqueue to node 192.168.201.133@tcp failed: rc = -12 Aug 15 06:14:44 oleg133-client sshd[26709]: Accepted publickey for root from 192.168.201.33 port 50770 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:44 oleg133-client systemd[1]: Started Session 123 of user root. Aug 15 06:14:44 oleg133-client systemd-logind[1034]: New session 123 of user root. Aug 15 06:14:44 oleg133-client sshd[26709]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:44 oleg133-client sshd[26723]: Received disconnect from 192.168.201.33 port 50770:11: disconnected by user Aug 15 06:14:44 oleg133-client sshd[26723]: Disconnected from user root 192.168.201.33 port 50770 Aug 15 06:14:44 oleg133-client sshd[26709]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:45 oleg133-client systemd-logind[1034]: Session 123 logged out. Waiting for processes to exit. Aug 15 06:14:45 oleg133-client systemd[1]: session-123.scope: Succeeded. Aug 15 06:14:45 oleg133-client systemd-logind[1034]: Removed session 123. Aug 15 06:14:46 oleg133-client sshd[26847]: Accepted publickey for root from 192.168.201.33 port 33398 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:46 oleg133-client systemd-logind[1034]: New session 124 of user root. Aug 15 06:14:46 oleg133-client systemd[1]: Started Session 124 of user root. Aug 15 06:14:46 oleg133-client sshd[26847]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:46 oleg133-client sshd[26860]: Received disconnect from 192.168.201.33 port 33398:11: disconnected by user Aug 15 06:14:46 oleg133-client sshd[26860]: Disconnected from user root 192.168.201.33 port 33398 Aug 15 06:14:46 oleg133-client sshd[26847]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:46 oleg133-client systemd[1]: session-124.scope: Succeeded. Aug 15 06:14:46 oleg133-client systemd-logind[1034]: Session 124 logged out. Waiting for processes to exit. Aug 15 06:14:46 oleg133-client systemd-logind[1034]: Removed session 124. Aug 15 06:14:48 oleg133-client sshd[26991]: Accepted publickey for root from 192.168.201.33 port 33408 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:48 oleg133-client systemd-logind[1034]: New session 125 of user root. Aug 15 06:14:48 oleg133-client systemd[1]: Started Session 125 of user root. Aug 15 06:14:48 oleg133-client sshd[26991]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:48 oleg133-client sshd[27004]: Received disconnect from 192.168.201.33 port 33408:11: disconnected by user Aug 15 06:14:48 oleg133-client sshd[27004]: Disconnected from user root 192.168.201.33 port 33408 Aug 15 06:14:48 oleg133-client sshd[26991]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:48 oleg133-client systemd[1]: session-125.scope: Succeeded. Aug 15 06:14:48 oleg133-client systemd-logind[1034]: Session 125 logged out. Waiting for processes to exit. Aug 15 06:14:48 oleg133-client systemd-logind[1034]: Removed session 125. Aug 15 06:14:49 oleg133-client sshd[27145]: Accepted publickey for root from 192.168.201.33 port 33424 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:14:50 oleg133-client systemd-logind[1034]: New session 126 of user root. Aug 15 06:14:50 oleg133-client systemd[1]: Started Session 126 of user root. Aug 15 06:14:50 oleg133-client sshd[27145]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:14:50 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 21a: drop close request while close and open are both in flight ========================================================== 06:14:49 (1786788889) Aug 15 06:14:50 oleg133-client sshd[27158]: Received disconnect from 192.168.201.33 port 33424:11: disconnected by user Aug 15 06:14:50 oleg133-client sshd[27158]: Disconnected from user root 192.168.201.33 port 33424 Aug 15 06:14:50 oleg133-client sshd[27145]: pam_unix(sshd:session): session closed for user root Aug 15 06:14:50 oleg133-client systemd[1]: session-126.scope: Succeeded. Aug 15 06:14:50 oleg133-client systemd-logind[1034]: Session 126 logged out. Waiting for processes to exit. Aug 15 06:14:50 oleg133-client systemd-logind[1034]: Removed session 126. Aug 15 06:15:15 oleg133-client sshd[27327]: Accepted publickey for root from 192.168.201.33 port 47214 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:15:15 oleg133-client systemd[1]: Started Session 127 of user root. Aug 15 06:15:15 oleg133-client systemd-logind[1034]: New session 127 of user root. Aug 15 06:15:15 oleg133-client sshd[27327]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:15:15 oleg133-client sshd[27340]: Received disconnect from 192.168.201.33 port 47214:11: disconnected by user Aug 15 06:15:15 oleg133-client sshd[27340]: Disconnected from user root 192.168.201.33 port 47214 Aug 15 06:15:15 oleg133-client sshd[27327]: pam_unix(sshd:session): session closed for user root Aug 15 06:15:15 oleg133-client systemd-logind[1034]: Session 127 logged out. Waiting for processes to exit. Aug 15 06:15:16 oleg133-client systemd[1]: session-127.scope: Succeeded. Aug 15 06:15:16 oleg133-client systemd-logind[1034]: Removed session 127. Aug 15 06:15:16 oleg133-client sshd[27465]: Accepted publickey for root from 192.168.201.33 port 41292 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:15:16 oleg133-client systemd-logind[1034]: New session 128 of user root. Aug 15 06:15:16 oleg133-client systemd[1]: Started Session 128 of user root. Aug 15 06:15:16 oleg133-client sshd[27465]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:15:17 oleg133-client sshd[27478]: Received disconnect from 192.168.201.33 port 41292:11: disconnected by user Aug 15 06:15:17 oleg133-client sshd[27478]: Disconnected from user root 192.168.201.33 port 41292 Aug 15 06:15:17 oleg133-client sshd[27465]: pam_unix(sshd:session): session closed for user root Aug 15 06:15:17 oleg133-client systemd[1]: session-128.scope: Succeeded. Aug 15 06:15:17 oleg133-client systemd-logind[1034]: Session 128 logged out. Waiting for processes to exit. Aug 15 06:15:17 oleg133-client systemd-logind[1034]: Removed session 128. Aug 15 06:15:18 oleg133-client sshd[27611]: Accepted publickey for root from 192.168.201.33 port 41296 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:15:18 oleg133-client systemd-logind[1034]: New session 129 of user root. Aug 15 06:15:18 oleg133-client systemd[1]: Started Session 129 of user root. Aug 15 06:15:18 oleg133-client sshd[27611]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:15:19 oleg133-client sshd[27624]: Received disconnect from 192.168.201.33 port 41296:11: disconnected by user Aug 15 06:15:19 oleg133-client sshd[27624]: Disconnected from user root 192.168.201.33 port 41296 Aug 15 06:15:19 oleg133-client sshd[27611]: pam_unix(sshd:session): session closed for user root Aug 15 06:15:19 oleg133-client systemd[1]: session-129.scope: Succeeded. Aug 15 06:15:19 oleg133-client systemd-logind[1034]: Session 129 logged out. Waiting for processes to exit. Aug 15 06:15:19 oleg133-client systemd-logind[1034]: Removed session 129. Aug 15 06:15:20 oleg133-client sshd[27764]: Accepted publickey for root from 192.168.201.33 port 41308 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:15:20 oleg133-client systemd[1]: Started Session 130 of user root. Aug 15 06:15:20 oleg133-client systemd-logind[1034]: New session 130 of user root. Aug 15 06:15:20 oleg133-client sshd[27764]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:15:20 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 21b: drop open request while close and open are both in flight ========================================================== 06:15:19 (1786788919) Aug 15 06:15:20 oleg133-client sshd[27777]: Received disconnect from 192.168.201.33 port 41308:11: disconnected by user Aug 15 06:15:20 oleg133-client sshd[27777]: Disconnected from user root 192.168.201.33 port 41308 Aug 15 06:15:20 oleg133-client sshd[27764]: pam_unix(sshd:session): session closed for user root Aug 15 06:15:20 oleg133-client systemd[1]: session-130.scope: Succeeded. Aug 15 06:15:20 oleg133-client systemd-logind[1034]: Session 130 logged out. Waiting for processes to exit. Aug 15 06:15:20 oleg133-client systemd-logind[1034]: Removed session 130. Aug 15 06:17:40 oleg133-client sshd[27947]: Accepted publickey for root from 192.168.201.33 port 42394 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:17:40 oleg133-client systemd-logind[1034]: New session 131 of user root. Aug 15 06:17:40 oleg133-client systemd[1]: Started Session 131 of user root. Aug 15 06:17:40 oleg133-client sshd[27947]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:17:40 oleg133-client sshd[27960]: Received disconnect from 192.168.201.33 port 42394:11: disconnected by user Aug 15 06:17:40 oleg133-client sshd[27960]: Disconnected from user root 192.168.201.33 port 42394 Aug 15 06:17:40 oleg133-client sshd[27947]: pam_unix(sshd:session): session closed for user root Aug 15 06:17:40 oleg133-client systemd-logind[1034]: Session 131 logged out. Waiting for processes to exit. Aug 15 06:17:40 oleg133-client systemd[1]: session-131.scope: Succeeded. Aug 15 06:17:40 oleg133-client systemd-logind[1034]: Removed session 131. Aug 15 06:17:42 oleg133-client sshd[28084]: Accepted publickey for root from 192.168.201.33 port 42396 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:17:42 oleg133-client systemd-logind[1034]: New session 132 of user root. Aug 15 06:17:42 oleg133-client systemd[1]: Started Session 132 of user root. Aug 15 06:17:42 oleg133-client sshd[28084]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:17:43 oleg133-client sshd[28097]: Received disconnect from 192.168.201.33 port 42396:11: disconnected by user Aug 15 06:17:43 oleg133-client sshd[28097]: Disconnected from user root 192.168.201.33 port 42396 Aug 15 06:17:43 oleg133-client sshd[28084]: pam_unix(sshd:session): session closed for user root Aug 15 06:17:43 oleg133-client systemd[1]: session-132.scope: Succeeded. Aug 15 06:17:43 oleg133-client systemd-logind[1034]: Session 132 logged out. Waiting for processes to exit. Aug 15 06:17:43 oleg133-client systemd-logind[1034]: Removed session 132. Aug 15 06:17:44 oleg133-client sshd[28230]: Accepted publickey for root from 192.168.201.33 port 42402 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:17:44 oleg133-client systemd-logind[1034]: New session 133 of user root. Aug 15 06:17:44 oleg133-client systemd[1]: Started Session 133 of user root. Aug 15 06:17:44 oleg133-client sshd[28230]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:17:44 oleg133-client sshd[28243]: Received disconnect from 192.168.201.33 port 42402:11: disconnected by user Aug 15 06:17:44 oleg133-client sshd[28243]: Disconnected from user root 192.168.201.33 port 42402 Aug 15 06:17:44 oleg133-client sshd[28230]: pam_unix(sshd:session): session closed for user root Aug 15 06:17:44 oleg133-client systemd[1]: session-133.scope: Succeeded. Aug 15 06:17:44 oleg133-client systemd-logind[1034]: Session 133 logged out. Waiting for processes to exit. Aug 15 06:17:45 oleg133-client systemd-logind[1034]: Removed session 133. Aug 15 06:17:46 oleg133-client sshd[28385]: Accepted publickey for root from 192.168.201.33 port 34858 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:17:46 oleg133-client systemd-logind[1034]: New session 134 of user root. Aug 15 06:17:46 oleg133-client systemd[1]: Started Session 134 of user root. Aug 15 06:17:46 oleg133-client sshd[28385]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:17:46 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 21c: drop both request while close and open are both in flight ========================================================== 06:17:45 (1786789065) Aug 15 06:17:46 oleg133-client sshd[28398]: Received disconnect from 192.168.201.33 port 34858:11: disconnected by user Aug 15 06:17:46 oleg133-client sshd[28398]: Disconnected from user root 192.168.201.33 port 34858 Aug 15 06:17:46 oleg133-client sshd[28385]: pam_unix(sshd:session): session closed for user root Aug 15 06:17:46 oleg133-client systemd-logind[1034]: Session 134 logged out. Waiting for processes to exit. Aug 15 06:17:46 oleg133-client systemd[1]: session-134.scope: Succeeded. Aug 15 06:17:46 oleg133-client systemd-logind[1034]: Removed session 134. Aug 15 06:18:09 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993744fd2000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete Aug 15 06:18:09 oleg133-client kernel: Lustre: Skipped 18 previous similar messages Aug 15 06:18:09 oleg133-client kernel: Lustre: lustre-MDT0000-mdc-ffff993744fd2000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) Aug 15 06:18:09 oleg133-client kernel: Lustre: Skipped 16 previous similar messages Aug 15 06:18:12 oleg133-client sshd[28568]: Accepted publickey for root from 192.168.201.33 port 54210 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:18:12 oleg133-client systemd-logind[1034]: New session 135 of user root. Aug 15 06:18:12 oleg133-client systemd[1]: Started Session 135 of user root. Aug 15 06:18:12 oleg133-client sshd[28568]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:18:13 oleg133-client sshd[28581]: Received disconnect from 192.168.201.33 port 54210:11: disconnected by user Aug 15 06:18:13 oleg133-client sshd[28581]: Disconnected from user root 192.168.201.33 port 54210 Aug 15 06:18:13 oleg133-client sshd[28568]: pam_unix(sshd:session): session closed for user root Aug 15 06:18:13 oleg133-client systemd[1]: session-135.scope: Succeeded. Aug 15 06:18:13 oleg133-client systemd-logind[1034]: Session 135 logged out. Waiting for processes to exit. Aug 15 06:18:13 oleg133-client systemd-logind[1034]: Removed session 135. Aug 15 06:18:14 oleg133-client sshd[28706]: Accepted publickey for root from 192.168.201.33 port 54220 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:18:14 oleg133-client systemd-logind[1034]: New session 136 of user root. Aug 15 06:18:14 oleg133-client systemd[1]: Started Session 136 of user root. Aug 15 06:18:14 oleg133-client sshd[28706]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:18:15 oleg133-client sshd[28719]: Received disconnect from 192.168.201.33 port 54220:11: disconnected by user Aug 15 06:18:15 oleg133-client sshd[28719]: Disconnected from user root 192.168.201.33 port 54220 Aug 15 06:18:15 oleg133-client sshd[28706]: pam_unix(sshd:session): session closed for user root Aug 15 06:18:15 oleg133-client systemd-logind[1034]: Session 136 logged out. Waiting for processes to exit. Aug 15 06:18:15 oleg133-client systemd[1]: session-136.scope: Succeeded. Aug 15 06:18:15 oleg133-client systemd-logind[1034]: Removed session 136. Aug 15 06:18:16 oleg133-client sshd[28850]: Accepted publickey for root from 192.168.201.33 port 42318 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:18:16 oleg133-client systemd-logind[1034]: New session 137 of user root. Aug 15 06:18:16 oleg133-client systemd[1]: Started Session 137 of user root. Aug 15 06:18:16 oleg133-client sshd[28850]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:18:16 oleg133-client sshd[28864]: Received disconnect from 192.168.201.33 port 42318:11: disconnected by user Aug 15 06:18:16 oleg133-client sshd[28864]: Disconnected from user root 192.168.201.33 port 42318 Aug 15 06:18:17 oleg133-client sshd[28850]: pam_unix(sshd:session): session closed for user root Aug 15 06:18:17 oleg133-client systemd-logind[1034]: Session 137 logged out. Waiting for processes to exit. Aug 15 06:18:17 oleg133-client systemd[1]: session-137.scope: Succeeded. Aug 15 06:18:17 oleg133-client systemd-logind[1034]: Removed session 137. Aug 15 06:18:18 oleg133-client sshd[29004]: Accepted publickey for root from 192.168.201.33 port 42330 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:18:18 oleg133-client systemd-logind[1034]: New session 138 of user root. Aug 15 06:18:18 oleg133-client systemd[1]: Started Session 138 of user root. Aug 15 06:18:18 oleg133-client sshd[29004]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:18:19 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 21d: drop close reply while close and open are both in flight ========================================================== 06:18:17 (1786789097) Aug 15 06:18:19 oleg133-client sshd[29017]: Received disconnect from 192.168.201.33 port 42330:11: disconnected by user Aug 15 06:18:19 oleg133-client sshd[29017]: Disconnected from user root 192.168.201.33 port 42330 Aug 15 06:18:19 oleg133-client sshd[29004]: pam_unix(sshd:session): session closed for user root Aug 15 06:18:19 oleg133-client systemd[1]: session-138.scope: Succeeded. Aug 15 06:18:19 oleg133-client systemd-logind[1034]: Session 138 logged out. Waiting for processes to exit. Aug 15 06:18:19 oleg133-client systemd-logind[1034]: Removed session 138. Aug 15 06:18:43 oleg133-client sshd[29187]: Accepted publickey for root from 192.168.201.33 port 52594 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:18:44 oleg133-client systemd-logind[1034]: New session 139 of user root. Aug 15 06:18:44 oleg133-client systemd[1]: Started Session 139 of user root. Aug 15 06:18:44 oleg133-client sshd[29187]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:18:44 oleg133-client sshd[29200]: Received disconnect from 192.168.201.33 port 52594:11: disconnected by user Aug 15 06:18:44 oleg133-client sshd[29200]: Disconnected from user root 192.168.201.33 port 52594 Aug 15 06:18:44 oleg133-client sshd[29187]: pam_unix(sshd:session): session closed for user root Aug 15 06:18:44 oleg133-client systemd[1]: session-139.scope: Succeeded. Aug 15 06:18:44 oleg133-client systemd-logind[1034]: Session 139 logged out. Waiting for processes to exit. Aug 15 06:18:44 oleg133-client systemd-logind[1034]: Removed session 139. Aug 15 06:18:45 oleg133-client sshd[29324]: Accepted publickey for root from 192.168.201.33 port 43716 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:18:45 oleg133-client systemd-logind[1034]: New session 140 of user root. Aug 15 06:18:45 oleg133-client systemd[1]: Started Session 140 of user root. Aug 15 06:18:45 oleg133-client sshd[29324]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:18:46 oleg133-client sshd[29337]: Received disconnect from 192.168.201.33 port 43716:11: disconnected by user Aug 15 06:18:46 oleg133-client sshd[29337]: Disconnected from user root 192.168.201.33 port 43716 Aug 15 06:18:46 oleg133-client sshd[29324]: pam_unix(sshd:session): session closed for user root Aug 15 06:18:46 oleg133-client systemd[1]: session-140.scope: Succeeded. Aug 15 06:18:46 oleg133-client systemd-logind[1034]: Session 140 logged out. Waiting for processes to exit. Aug 15 06:18:46 oleg133-client systemd-logind[1034]: Removed session 140. Aug 15 06:18:47 oleg133-client sshd[29469]: Accepted publickey for root from 192.168.201.33 port 43722 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:18:47 oleg133-client systemd-logind[1034]: New session 141 of user root. Aug 15 06:18:47 oleg133-client systemd[1]: Started Session 141 of user root. Aug 15 06:18:47 oleg133-client sshd[29469]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:18:48 oleg133-client sshd[29482]: Received disconnect from 192.168.201.33 port 43722:11: disconnected by user Aug 15 06:18:48 oleg133-client sshd[29482]: Disconnected from user root 192.168.201.33 port 43722 Aug 15 06:18:48 oleg133-client sshd[29469]: pam_unix(sshd:session): session closed for user root Aug 15 06:18:48 oleg133-client systemd-logind[1034]: Session 141 logged out. Waiting for processes to exit. Aug 15 06:18:48 oleg133-client systemd[1]: session-141.scope: Succeeded. Aug 15 06:18:48 oleg133-client systemd-logind[1034]: Removed session 141. Aug 15 06:18:50 oleg133-client sshd[29622]: Accepted publickey for root from 192.168.201.33 port 43724 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:18:50 oleg133-client systemd-logind[1034]: New session 142 of user root. Aug 15 06:18:50 oleg133-client systemd[1]: Started Session 142 of user root. Aug 15 06:18:50 oleg133-client sshd[29622]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:18:50 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 21e: drop open reply while close and open are both in flight ========================================================== 06:18:48 (1786789128) Aug 15 06:18:50 oleg133-client sshd[29635]: Received disconnect from 192.168.201.33 port 43724:11: disconnected by user Aug 15 06:18:50 oleg133-client sshd[29635]: Disconnected from user root 192.168.201.33 port 43724 Aug 15 06:18:50 oleg133-client sshd[29622]: pam_unix(sshd:session): session closed for user root Aug 15 06:18:51 oleg133-client systemd[1]: session-142.scope: Succeeded. Aug 15 06:18:51 oleg133-client systemd-logind[1034]: Session 142 logged out. Waiting for processes to exit. Aug 15 06:18:51 oleg133-client systemd-logind[1034]: Removed session 142. Aug 15 06:19:16 oleg133-client sshd[29789]: Accepted publickey for root from 192.168.201.33 port 56188 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:19:16 oleg133-client systemd-logind[1034]: New session 143 of user root. Aug 15 06:19:16 oleg133-client systemd[1]: Started Session 143 of user root. Aug 15 06:19:16 oleg133-client sshd[29789]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:19:16 oleg133-client sshd[29802]: Received disconnect from 192.168.201.33 port 56188:11: disconnected by user Aug 15 06:19:16 oleg133-client sshd[29802]: Disconnected from user root 192.168.201.33 port 56188 Aug 15 06:19:16 oleg133-client sshd[29789]: pam_unix(sshd:session): session closed for user root Aug 15 06:19:17 oleg133-client systemd-logind[1034]: Session 143 logged out. Waiting for processes to exit. Aug 15 06:19:17 oleg133-client systemd[1]: session-143.scope: Succeeded. Aug 15 06:19:17 oleg133-client systemd-logind[1034]: Removed session 143. Aug 15 06:19:19 oleg133-client sshd[29927]: Accepted publickey for root from 192.168.201.33 port 56204 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:19:19 oleg133-client systemd-logind[1034]: New session 144 of user root. Aug 15 06:19:19 oleg133-client systemd[1]: Started Session 144 of user root. Aug 15 06:19:19 oleg133-client sshd[29927]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:19:19 oleg133-client sshd[29940]: Received disconnect from 192.168.201.33 port 56204:11: disconnected by user Aug 15 06:19:19 oleg133-client sshd[29940]: Disconnected from user root 192.168.201.33 port 56204 Aug 15 06:19:19 oleg133-client sshd[29927]: pam_unix(sshd:session): session closed for user root Aug 15 06:19:20 oleg133-client systemd-logind[1034]: Session 144 logged out. Waiting for processes to exit. Aug 15 06:19:20 oleg133-client systemd[1]: session-144.scope: Succeeded. Aug 15 06:19:20 oleg133-client systemd-logind[1034]: Removed session 144. Aug 15 06:19:21 oleg133-client sshd[30071]: Accepted publickey for root from 192.168.201.33 port 56210 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:19:21 oleg133-client systemd[1]: Started Session 145 of user root. Aug 15 06:19:21 oleg133-client systemd-logind[1034]: New session 145 of user root. Aug 15 06:19:21 oleg133-client sshd[30071]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:19:21 oleg133-client sshd[30084]: Received disconnect from 192.168.201.33 port 56210:11: disconnected by user Aug 15 06:19:21 oleg133-client sshd[30084]: Disconnected from user root 192.168.201.33 port 56210 Aug 15 06:19:21 oleg133-client sshd[30071]: pam_unix(sshd:session): session closed for user root Aug 15 06:19:21 oleg133-client systemd-logind[1034]: Session 145 logged out. Waiting for processes to exit. Aug 15 06:19:22 oleg133-client systemd[1]: session-145.scope: Succeeded. Aug 15 06:19:22 oleg133-client systemd-logind[1034]: Removed session 145. Aug 15 06:21:08 oleg133-client kernel: Lustre: 29683:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786789132/real 1786789132] req@ffff993760343480 x1873582575862400/t0(0) o36->lustre-MDT0000-mdc-ffff993744fd2000@192.168.201.133@tcp:12/10 lens 488/512 e 0 to 1 dl 1786789268 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:4294967295 Aug 15 06:21:08 oleg133-client kernel: Lustre: 29683:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 14 previous similar messages Aug 15 06:21:10 oleg133-client sshd[30240]: Accepted publickey for root from 192.168.201.33 port 41966 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:21:10 oleg133-client systemd-logind[1034]: New session 146 of user root. Aug 15 06:21:10 oleg133-client systemd[1]: Started Session 146 of user root. Aug 15 06:21:10 oleg133-client sshd[30240]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:21:10 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 21f: drop both reply while close and open are both in flight ========================================================== 06:21:09 (1786789269) Aug 15 06:21:10 oleg133-client sshd[30253]: Received disconnect from 192.168.201.33 port 41966:11: disconnected by user Aug 15 06:21:10 oleg133-client sshd[30253]: Disconnected from user root 192.168.201.33 port 41966 Aug 15 06:21:10 oleg133-client sshd[30240]: pam_unix(sshd:session): session closed for user root Aug 15 06:21:11 oleg133-client systemd[1]: session-146.scope: Succeeded. Aug 15 06:21:11 oleg133-client systemd-logind[1034]: Session 146 logged out. Waiting for processes to exit. Aug 15 06:21:11 oleg133-client systemd-logind[1034]: Removed session 146. Aug 15 06:21:32 oleg133-client sshd[30423]: Accepted publickey for root from 192.168.201.33 port 49636 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:21:32 oleg133-client systemd-logind[1034]: New session 147 of user root. Aug 15 06:21:32 oleg133-client systemd[1]: Started Session 147 of user root. Aug 15 06:21:32 oleg133-client sshd[30423]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:21:33 oleg133-client sshd[30436]: Received disconnect from 192.168.201.33 port 49636:11: disconnected by user Aug 15 06:21:33 oleg133-client sshd[30436]: Disconnected from user root 192.168.201.33 port 49636 Aug 15 06:21:33 oleg133-client sshd[30423]: pam_unix(sshd:session): session closed for user root Aug 15 06:21:33 oleg133-client systemd-logind[1034]: Session 147 logged out. Waiting for processes to exit. Aug 15 06:21:33 oleg133-client systemd[1]: session-147.scope: Succeeded. Aug 15 06:21:33 oleg133-client systemd-logind[1034]: Removed session 147. Aug 15 06:21:34 oleg133-client sshd[30561]: Accepted publickey for root from 192.168.201.33 port 49642 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:21:34 oleg133-client systemd-logind[1034]: New session 148 of user root. Aug 15 06:21:34 oleg133-client systemd[1]: Started Session 148 of user root. Aug 15 06:21:34 oleg133-client sshd[30561]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:21:35 oleg133-client sshd[30574]: Received disconnect from 192.168.201.33 port 49642:11: disconnected by user Aug 15 06:21:35 oleg133-client sshd[30574]: Disconnected from user root 192.168.201.33 port 49642 Aug 15 06:21:35 oleg133-client sshd[30561]: pam_unix(sshd:session): session closed for user root Aug 15 06:21:35 oleg133-client systemd[1]: session-148.scope: Succeeded. Aug 15 06:21:35 oleg133-client systemd-logind[1034]: Session 148 logged out. Waiting for processes to exit. Aug 15 06:21:35 oleg133-client systemd-logind[1034]: Removed session 148. Aug 15 06:21:36 oleg133-client sshd[30705]: Accepted publickey for root from 192.168.201.33 port 55222 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:21:36 oleg133-client systemd-logind[1034]: New session 149 of user root. Aug 15 06:21:36 oleg133-client systemd[1]: Started Session 149 of user root. Aug 15 06:21:36 oleg133-client sshd[30705]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:21:36 oleg133-client sshd[30718]: Received disconnect from 192.168.201.33 port 55222:11: disconnected by user Aug 15 06:21:36 oleg133-client sshd[30718]: Disconnected from user root 192.168.201.33 port 55222 Aug 15 06:21:36 oleg133-client sshd[30705]: pam_unix(sshd:session): session closed for user root Aug 15 06:21:36 oleg133-client systemd[1]: session-149.scope: Succeeded. Aug 15 06:21:36 oleg133-client systemd-logind[1034]: Session 149 logged out. Waiting for processes to exit. Aug 15 06:21:36 oleg133-client systemd-logind[1034]: Removed session 149. Aug 15 06:21:37 oleg133-client sshd[30858]: Accepted publickey for root from 192.168.201.33 port 55238 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:21:37 oleg133-client systemd-logind[1034]: New session 150 of user root. Aug 15 06:21:37 oleg133-client systemd[1]: Started Session 150 of user root. Aug 15 06:21:37 oleg133-client sshd[30858]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:21:38 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 21g: drop open reply and close request while close and open are both in flight ========================================================== 06:21:37 (1786789297) Aug 15 06:21:38 oleg133-client sshd[30871]: Received disconnect from 192.168.201.33 port 55238:11: disconnected by user Aug 15 06:21:38 oleg133-client sshd[30871]: Disconnected from user root 192.168.201.33 port 55238 Aug 15 06:21:38 oleg133-client sshd[30858]: pam_unix(sshd:session): session closed for user root Aug 15 06:21:38 oleg133-client systemd[1]: session-150.scope: Succeeded. Aug 15 06:21:38 oleg133-client systemd-logind[1034]: Session 150 logged out. Waiting for processes to exit. Aug 15 06:21:38 oleg133-client systemd-logind[1034]: Removed session 150. Aug 15 06:21:59 oleg133-client sshd[31040]: Accepted publickey for root from 192.168.201.33 port 46630 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:21:59 oleg133-client systemd-logind[1034]: New session 151 of user root. Aug 15 06:21:59 oleg133-client systemd[1]: Started Session 151 of user root. Aug 15 06:21:59 oleg133-client sshd[31040]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:00 oleg133-client sshd[31054]: Received disconnect from 192.168.201.33 port 46630:11: disconnected by user Aug 15 06:22:00 oleg133-client sshd[31054]: Disconnected from user root 192.168.201.33 port 46630 Aug 15 06:22:00 oleg133-client sshd[31040]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:00 oleg133-client systemd[1]: session-151.scope: Succeeded. Aug 15 06:22:00 oleg133-client systemd-logind[1034]: Session 151 logged out. Waiting for processes to exit. Aug 15 06:22:00 oleg133-client systemd-logind[1034]: Removed session 151. Aug 15 06:22:01 oleg133-client sshd[31178]: Accepted publickey for root from 192.168.201.33 port 46644 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:01 oleg133-client systemd-logind[1034]: New session 152 of user root. Aug 15 06:22:01 oleg133-client systemd[1]: Started Session 152 of user root. Aug 15 06:22:01 oleg133-client sshd[31178]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:02 oleg133-client sshd[31191]: Received disconnect from 192.168.201.33 port 46644:11: disconnected by user Aug 15 06:22:02 oleg133-client sshd[31191]: Disconnected from user root 192.168.201.33 port 46644 Aug 15 06:22:02 oleg133-client sshd[31178]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:02 oleg133-client systemd[1]: session-152.scope: Succeeded. Aug 15 06:22:02 oleg133-client systemd-logind[1034]: Session 152 logged out. Waiting for processes to exit. Aug 15 06:22:02 oleg133-client systemd-logind[1034]: Removed session 152. Aug 15 06:22:03 oleg133-client sshd[31322]: Accepted publickey for root from 192.168.201.33 port 46648 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:03 oleg133-client systemd[1]: Started Session 153 of user root. Aug 15 06:22:03 oleg133-client systemd-logind[1034]: New session 153 of user root. Aug 15 06:22:03 oleg133-client sshd[31322]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:03 oleg133-client sshd[31335]: Received disconnect from 192.168.201.33 port 46648:11: disconnected by user Aug 15 06:22:03 oleg133-client sshd[31335]: Disconnected from user root 192.168.201.33 port 46648 Aug 15 06:22:03 oleg133-client sshd[31322]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:03 oleg133-client systemd[1]: session-153.scope: Succeeded. Aug 15 06:22:03 oleg133-client systemd-logind[1034]: Session 153 logged out. Waiting for processes to exit. Aug 15 06:22:03 oleg133-client systemd-logind[1034]: Removed session 153. Aug 15 06:22:05 oleg133-client sshd[31475]: Accepted publickey for root from 192.168.201.33 port 46664 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:05 oleg133-client systemd-logind[1034]: New session 154 of user root. Aug 15 06:22:05 oleg133-client systemd[1]: Started Session 154 of user root. Aug 15 06:22:05 oleg133-client sshd[31475]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:05 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 21h: drop open request and close reply while close and open are both in flight ========================================================== 06:22:04 (1786789324) Aug 15 06:22:05 oleg133-client sshd[31488]: Received disconnect from 192.168.201.33 port 46664:11: disconnected by user Aug 15 06:22:06 oleg133-client sshd[31488]: Disconnected from user root 192.168.201.33 port 46664 Aug 15 06:22:06 oleg133-client sshd[31475]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:06 oleg133-client systemd-logind[1034]: Session 154 logged out. Waiting for processes to exit. Aug 15 06:22:06 oleg133-client systemd[1]: session-154.scope: Succeeded. Aug 15 06:22:06 oleg133-client systemd-logind[1034]: Removed session 154. Aug 15 06:22:26 oleg133-client sshd[31658]: Accepted publickey for root from 192.168.201.33 port 58350 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:26 oleg133-client systemd-logind[1034]: New session 155 of user root. Aug 15 06:22:26 oleg133-client systemd[1]: Started Session 155 of user root. Aug 15 06:22:26 oleg133-client sshd[31658]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:27 oleg133-client sshd[31671]: Received disconnect from 192.168.201.33 port 58350:11: disconnected by user Aug 15 06:22:27 oleg133-client sshd[31671]: Disconnected from user root 192.168.201.33 port 58350 Aug 15 06:22:27 oleg133-client sshd[31658]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:27 oleg133-client systemd[1]: session-155.scope: Succeeded. Aug 15 06:22:27 oleg133-client systemd-logind[1034]: Session 155 logged out. Waiting for processes to exit. Aug 15 06:22:27 oleg133-client systemd-logind[1034]: Removed session 155. Aug 15 06:22:28 oleg133-client sshd[31795]: Accepted publickey for root from 192.168.201.33 port 58352 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:28 oleg133-client systemd-logind[1034]: New session 156 of user root. Aug 15 06:22:28 oleg133-client systemd[1]: Started Session 156 of user root. Aug 15 06:22:28 oleg133-client sshd[31795]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:28 oleg133-client sshd[31808]: Received disconnect from 192.168.201.33 port 58352:11: disconnected by user Aug 15 06:22:28 oleg133-client sshd[31808]: Disconnected from user root 192.168.201.33 port 58352 Aug 15 06:22:28 oleg133-client sshd[31795]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:28 oleg133-client systemd-logind[1034]: Session 156 logged out. Waiting for processes to exit. Aug 15 06:22:28 oleg133-client systemd[1]: session-156.scope: Succeeded. Aug 15 06:22:28 oleg133-client systemd-logind[1034]: Removed session 156. Aug 15 06:22:29 oleg133-client sshd[31940]: Accepted publickey for root from 192.168.201.33 port 58354 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:29 oleg133-client systemd[1]: Started Session 157 of user root. Aug 15 06:22:29 oleg133-client systemd-logind[1034]: New session 157 of user root. Aug 15 06:22:29 oleg133-client sshd[31940]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:29 oleg133-client sshd[31953]: Received disconnect from 192.168.201.33 port 58354:11: disconnected by user Aug 15 06:22:29 oleg133-client sshd[31953]: Disconnected from user root 192.168.201.33 port 58354 Aug 15 06:22:29 oleg133-client sshd[31940]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:30 oleg133-client systemd[1]: session-157.scope: Succeeded. Aug 15 06:22:30 oleg133-client systemd-logind[1034]: Session 157 logged out. Waiting for processes to exit. Aug 15 06:22:30 oleg133-client systemd-logind[1034]: Removed session 157. Aug 15 06:22:30 oleg133-client sshd[32094]: Accepted publickey for root from 192.168.201.33 port 58362 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:30 oleg133-client systemd[1]: Started Session 158 of user root. Aug 15 06:22:30 oleg133-client sshd[32094]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:30 oleg133-client systemd-logind[1034]: New session 158 of user root. Aug 15 06:22:31 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 22: drop close request and do mknod ========================================================== 06:22:30 (1786789350) Aug 15 06:22:31 oleg133-client sshd[32107]: Received disconnect from 192.168.201.33 port 58362:11: disconnected by user Aug 15 06:22:31 oleg133-client sshd[32107]: Disconnected from user root 192.168.201.33 port 58362 Aug 15 06:22:31 oleg133-client sshd[32094]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:31 oleg133-client systemd-logind[1034]: Session 158 logged out. Waiting for processes to exit. Aug 15 06:22:31 oleg133-client systemd[1]: session-158.scope: Succeeded. Aug 15 06:22:31 oleg133-client systemd-logind[1034]: Removed session 158. Aug 15 06:22:50 oleg133-client sshd[32255]: Accepted publickey for root from 192.168.201.33 port 60744 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:50 oleg133-client systemd-logind[1034]: New session 159 of user root. Aug 15 06:22:50 oleg133-client systemd[1]: Started Session 159 of user root. Aug 15 06:22:50 oleg133-client sshd[32255]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:50 oleg133-client sshd[32268]: Received disconnect from 192.168.201.33 port 60744:11: disconnected by user Aug 15 06:22:50 oleg133-client sshd[32268]: Disconnected from user root 192.168.201.33 port 60744 Aug 15 06:22:50 oleg133-client sshd[32255]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:50 oleg133-client systemd-logind[1034]: Session 159 logged out. Waiting for processes to exit. Aug 15 06:22:50 oleg133-client systemd[1]: session-159.scope: Succeeded. Aug 15 06:22:50 oleg133-client systemd-logind[1034]: Removed session 159. Aug 15 06:22:52 oleg133-client sshd[32393]: Accepted publickey for root from 192.168.201.33 port 60748 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:52 oleg133-client systemd-logind[1034]: New session 160 of user root. Aug 15 06:22:52 oleg133-client systemd[1]: Started Session 160 of user root. Aug 15 06:22:52 oleg133-client sshd[32393]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:53 oleg133-client sshd[32406]: Received disconnect from 192.168.201.33 port 60748:11: disconnected by user Aug 15 06:22:53 oleg133-client sshd[32406]: Disconnected from user root 192.168.201.33 port 60748 Aug 15 06:22:53 oleg133-client sshd[32393]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:53 oleg133-client systemd-logind[1034]: Session 160 logged out. Waiting for processes to exit. Aug 15 06:22:53 oleg133-client systemd[1]: session-160.scope: Succeeded. Aug 15 06:22:53 oleg133-client systemd-logind[1034]: Removed session 160. Aug 15 06:22:54 oleg133-client sshd[32537]: Accepted publickey for root from 192.168.201.33 port 60754 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:54 oleg133-client systemd-logind[1034]: New session 161 of user root. Aug 15 06:22:54 oleg133-client systemd[1]: Started Session 161 of user root. Aug 15 06:22:54 oleg133-client sshd[32537]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:54 oleg133-client sshd[32550]: Received disconnect from 192.168.201.33 port 60754:11: disconnected by user Aug 15 06:22:55 oleg133-client sshd[32550]: Disconnected from user root 192.168.201.33 port 60754 Aug 15 06:22:55 oleg133-client sshd[32537]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:55 oleg133-client systemd-logind[1034]: Session 161 logged out. Waiting for processes to exit. Aug 15 06:22:55 oleg133-client systemd[1]: session-161.scope: Succeeded. Aug 15 06:22:55 oleg133-client systemd-logind[1034]: Removed session 161. Aug 15 06:22:56 oleg133-client sshd[32690]: Accepted publickey for root from 192.168.201.33 port 34264 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:22:56 oleg133-client systemd[1]: Started Session 162 of user root. Aug 15 06:22:56 oleg133-client systemd-logind[1034]: New session 162 of user root. Aug 15 06:22:56 oleg133-client sshd[32690]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:22:56 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 23: client hang when close a file after mds crash ========================================================== 06:22:55 (1786789375) Aug 15 06:22:56 oleg133-client sshd[32703]: Received disconnect from 192.168.201.33 port 34264:11: disconnected by user Aug 15 06:22:56 oleg133-client sshd[32703]: Disconnected from user root 192.168.201.33 port 34264 Aug 15 06:22:56 oleg133-client sshd[32690]: pam_unix(sshd:session): session closed for user root Aug 15 06:22:56 oleg133-client systemd[1]: session-162.scope: Succeeded. Aug 15 06:22:56 oleg133-client systemd-logind[1034]: Session 162 logged out. Waiting for processes to exit. Aug 15 06:22:56 oleg133-client systemd-logind[1034]: Removed session 162. Aug 15 06:23:24 oleg133-client kernel: LustreError: MGC192.168.201.133@tcp: Connection to MGS (at 192.168.201.133@tcp) was lost; in progress operations using this service will fail Aug 15 06:23:34 oleg133-client kernel: Lustre: Evicted from MGS (at 192.168.201.133@tcp) after server handle changed from 0x708664883ac6d383 to 0x708664883ac6f82e Aug 15 06:23:35 oleg133-client kernel: Lustre: 15962:0:(mgc_request.c:1899:mgc_process_log()) MGC192.168.201.133@tcp: IR log lustre-cliir failed, not fatal: rc = -2 Aug 15 06:23:48 oleg133-client sshd[33046]: Accepted publickey for root from 192.168.201.33 port 51210 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:23:48 oleg133-client systemd-logind[1034]: New session 163 of user root. Aug 15 06:23:48 oleg133-client systemd[1]: Started Session 163 of user root. Aug 15 06:23:48 oleg133-client sshd[33046]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:23:49 oleg133-client sshd[33059]: Received disconnect from 192.168.201.33 port 51210:11: disconnected by user Aug 15 06:23:49 oleg133-client sshd[33059]: Disconnected from user root 192.168.201.33 port 51210 Aug 15 06:23:49 oleg133-client sshd[33046]: pam_unix(sshd:session): session closed for user root Aug 15 06:23:49 oleg133-client systemd-logind[1034]: Session 163 logged out. Waiting for processes to exit. Aug 15 06:23:49 oleg133-client systemd[1]: session-163.scope: Succeeded. Aug 15 06:23:49 oleg133-client systemd-logind[1034]: Removed session 163. Aug 15 06:23:54 oleg133-client sshd[33308]: Accepted publickey for root from 192.168.201.33 port 51218 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:23:54 oleg133-client systemd[1]: Started Session 164 of user root. Aug 15 06:23:54 oleg133-client systemd-logind[1034]: New session 164 of user root. Aug 15 06:23:54 oleg133-client sshd[33308]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:23:55 oleg133-client kernel: Lustre: DEBUG MARKER: oleg133-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 Aug 15 06:23:55 oleg133-client sshd[33321]: Received disconnect from 192.168.201.33 port 51218:11: disconnected by user Aug 15 06:23:55 oleg133-client sshd[33321]: Disconnected from user root 192.168.201.33 port 51218 Aug 15 06:23:55 oleg133-client sshd[33308]: pam_unix(sshd:session): session closed for user root Aug 15 06:23:55 oleg133-client systemd[1]: session-164.scope: Succeeded. Aug 15 06:23:55 oleg133-client systemd-logind[1034]: Session 164 logged out. Waiting for processes to exit. Aug 15 06:23:55 oleg133-client systemd-logind[1034]: Removed session 164. Aug 15 06:23:56 oleg133-client sshd[33455]: Accepted publickey for root from 192.168.201.33 port 44472 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:23:56 oleg133-client systemd[1]: Started Session 165 of user root. Aug 15 06:23:56 oleg133-client systemd-logind[1034]: New session 165 of user root. Aug 15 06:23:56 oleg133-client sshd[33455]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:23:57 oleg133-client kernel: Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Aug 15 06:23:57 oleg133-client sshd[33468]: Received disconnect from 192.168.201.33 port 44472:11: disconnected by user Aug 15 06:23:57 oleg133-client sshd[33468]: Disconnected from user root 192.168.201.33 port 44472 Aug 15 06:23:57 oleg133-client sshd[33455]: pam_unix(sshd:session): session closed for user root Aug 15 06:23:57 oleg133-client systemd[1]: session-165.scope: Succeeded. Aug 15 06:23:57 oleg133-client systemd-logind[1034]: Session 165 logged out. Waiting for processes to exit. Aug 15 06:23:57 oleg133-client systemd-logind[1034]: Removed session 165. Aug 15 06:23:59 oleg133-client sshd[33509]: Accepted publickey for root from 192.168.201.33 port 44476 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:23:59 oleg133-client systemd-logind[1034]: New session 166 of user root. Aug 15 06:23:59 oleg133-client systemd[1]: Started Session 166 of user root. Aug 15 06:23:59 oleg133-client sshd[33509]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:23:59 oleg133-client sshd[33522]: Received disconnect from 192.168.201.33 port 44476:11: disconnected by user Aug 15 06:23:59 oleg133-client sshd[33522]: Disconnected from user root 192.168.201.33 port 44476 Aug 15 06:23:59 oleg133-client sshd[33509]: pam_unix(sshd:session): session closed for user root Aug 15 06:23:59 oleg133-client systemd[1]: session-166.scope: Succeeded. Aug 15 06:23:59 oleg133-client systemd-logind[1034]: Session 166 logged out. Waiting for processes to exit. Aug 15 06:23:59 oleg133-client systemd-logind[1034]: Removed session 166. Aug 15 06:24:00 oleg133-client sshd[33640]: Accepted publickey for root from 192.168.201.33 port 44478 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:00 oleg133-client systemd-logind[1034]: New session 167 of user root. Aug 15 06:24:00 oleg133-client systemd[1]: Started Session 167 of user root. Aug 15 06:24:00 oleg133-client sshd[33640]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:01 oleg133-client sshd[33653]: Received disconnect from 192.168.201.33 port 44478:11: disconnected by user Aug 15 06:24:01 oleg133-client sshd[33653]: Disconnected from user root 192.168.201.33 port 44478 Aug 15 06:24:01 oleg133-client sshd[33640]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:01 oleg133-client systemd-logind[1034]: Session 167 logged out. Waiting for processes to exit. Aug 15 06:24:01 oleg133-client systemd[1]: session-167.scope: Succeeded. Aug 15 06:24:01 oleg133-client systemd-logind[1034]: Removed session 167. Aug 15 06:24:02 oleg133-client sshd[33778]: Accepted publickey for root from 192.168.201.33 port 44490 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:02 oleg133-client systemd-logind[1034]: New session 168 of user root. Aug 15 06:24:02 oleg133-client systemd[1]: Started Session 168 of user root. Aug 15 06:24:02 oleg133-client sshd[33778]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:02 oleg133-client sshd[33792]: Received disconnect from 192.168.201.33 port 44490:11: disconnected by user Aug 15 06:24:02 oleg133-client sshd[33792]: Disconnected from user root 192.168.201.33 port 44490 Aug 15 06:24:02 oleg133-client sshd[33778]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:02 oleg133-client systemd-logind[1034]: Session 168 logged out. Waiting for processes to exit. Aug 15 06:24:02 oleg133-client systemd[1]: session-168.scope: Succeeded. Aug 15 06:24:02 oleg133-client systemd-logind[1034]: Removed session 168. Aug 15 06:24:04 oleg133-client sshd[33923]: Accepted publickey for root from 192.168.201.33 port 44506 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:04 oleg133-client systemd-logind[1034]: New session 169 of user root. Aug 15 06:24:04 oleg133-client systemd[1]: Started Session 169 of user root. Aug 15 06:24:04 oleg133-client sshd[33923]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:04 oleg133-client sshd[33936]: Received disconnect from 192.168.201.33 port 44506:11: disconnected by user Aug 15 06:24:04 oleg133-client sshd[33936]: Disconnected from user root 192.168.201.33 port 44506 Aug 15 06:24:04 oleg133-client sshd[33923]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:04 oleg133-client systemd[1]: session-169.scope: Succeeded. Aug 15 06:24:04 oleg133-client systemd-logind[1034]: Session 169 logged out. Waiting for processes to exit. Aug 15 06:24:04 oleg133-client systemd-logind[1034]: Removed session 169. Aug 15 06:24:06 oleg133-client sshd[34076]: Accepted publickey for root from 192.168.201.33 port 40600 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:06 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:24:06 oleg133-client systemd-logind[1034]: New session 170 of user root. Aug 15 06:24:06 oleg133-client systemd[1]: Started Session 170 of user root. Aug 15 06:24:06 oleg133-client sshd[34076]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:06 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 24a: fsync error (should return error) ========================================================== 06:24:05 (1786789445) Aug 15 06:24:06 oleg133-client sshd[34090]: Received disconnect from 192.168.201.33 port 40600:11: disconnected by user Aug 15 06:24:06 oleg133-client sshd[34090]: Disconnected from user root 192.168.201.33 port 40600 Aug 15 06:24:06 oleg133-client sshd[34076]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:06 oleg133-client systemd[1]: session-170.scope: Succeeded. Aug 15 06:24:06 oleg133-client systemd-logind[1034]: Session 170 logged out. Waiting for processes to exit. Aug 15 06:24:06 oleg133-client systemd-logind[1034]: Removed session 170. Aug 15 06:24:08 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: operation ost_write to node 192.168.201.133@tcp failed: rc = -107 Aug 15 06:24:08 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. Aug 15 06:24:08 oleg133-client kernel: Lustre: 2376:0:(llite_lib.c:4340:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.201.133@tcp:/lustre/fid: [0x200000404:0x30:0x0]// may get corrupted (rc -5) Aug 15 06:24:08 oleg133-client kernel: LustreError: 34177:0:(ldlm_resource.c:1207:ldlm_resource_complain()) lustre-OST0000-osc-ffff993744fd2000: namespace resource [0x240000400:0x42:0x0].0x0 (ffff993742dbfb00) refcount nonzero (1) after lock cleanup; forcing cleanup. Aug 15 06:24:10 oleg133-client sshd[34274]: Accepted publickey for root from 192.168.201.33 port 40608 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:10 oleg133-client systemd-logind[1034]: New session 171 of user root. Aug 15 06:24:10 oleg133-client systemd[1]: Started Session 171 of user root. Aug 15 06:24:10 oleg133-client sshd[34274]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:10 oleg133-client sshd[34287]: Received disconnect from 192.168.201.33 port 40608:11: disconnected by user Aug 15 06:24:10 oleg133-client sshd[34287]: Disconnected from user root 192.168.201.33 port 40608 Aug 15 06:24:10 oleg133-client sshd[34274]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:10 oleg133-client systemd-logind[1034]: Session 171 logged out. Waiting for processes to exit. Aug 15 06:24:11 oleg133-client systemd[1]: session-171.scope: Succeeded. Aug 15 06:24:11 oleg133-client systemd-logind[1034]: Removed session 171. Aug 15 06:24:12 oleg133-client sshd[34411]: Accepted publickey for root from 192.168.201.33 port 40622 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:12 oleg133-client systemd-logind[1034]: New session 172 of user root. Aug 15 06:24:12 oleg133-client systemd[1]: Started Session 172 of user root. Aug 15 06:24:12 oleg133-client sshd[34411]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:13 oleg133-client sshd[34424]: Received disconnect from 192.168.201.33 port 40622:11: disconnected by user Aug 15 06:24:13 oleg133-client sshd[34424]: Disconnected from user root 192.168.201.33 port 40622 Aug 15 06:24:13 oleg133-client sshd[34411]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:13 oleg133-client systemd-logind[1034]: Session 172 logged out. Waiting for processes to exit. Aug 15 06:24:13 oleg133-client systemd[1]: session-172.scope: Succeeded. Aug 15 06:24:13 oleg133-client systemd-logind[1034]: Removed session 172. Aug 15 06:24:14 oleg133-client sshd[34555]: Accepted publickey for root from 192.168.201.33 port 40634 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:14 oleg133-client systemd-logind[1034]: New session 173 of user root. Aug 15 06:24:14 oleg133-client systemd[1]: Started Session 173 of user root. Aug 15 06:24:14 oleg133-client sshd[34555]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:15 oleg133-client sshd[34568]: Received disconnect from 192.168.201.33 port 40634:11: disconnected by user Aug 15 06:24:15 oleg133-client sshd[34568]: Disconnected from user root 192.168.201.33 port 40634 Aug 15 06:24:15 oleg133-client sshd[34555]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:15 oleg133-client systemd-logind[1034]: Session 173 logged out. Waiting for processes to exit. Aug 15 06:24:15 oleg133-client systemd[1]: session-173.scope: Succeeded. Aug 15 06:24:15 oleg133-client systemd-logind[1034]: Removed session 173. Aug 15 06:24:16 oleg133-client sshd[34709]: Accepted publickey for root from 192.168.201.33 port 39392 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:16 oleg133-client systemd-logind[1034]: New session 174 of user root. Aug 15 06:24:16 oleg133-client systemd[1]: Started Session 174 of user root. Aug 15 06:24:16 oleg133-client sshd[34709]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:17 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 24b: test dirty page discard due to client eviction ========================================================== 06:24:15 (1786789455) Aug 15 06:24:17 oleg133-client sshd[34722]: Received disconnect from 192.168.201.33 port 39392:11: disconnected by user Aug 15 06:24:17 oleg133-client sshd[34722]: Disconnected from user root 192.168.201.33 port 39392 Aug 15 06:24:17 oleg133-client sshd[34709]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:17 oleg133-client systemd-logind[1034]: Session 174 logged out. Waiting for processes to exit. Aug 15 06:24:17 oleg133-client systemd[1]: session-174.scope: Succeeded. Aug 15 06:24:17 oleg133-client systemd-logind[1034]: Removed session 174. Aug 15 06:24:19 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: operation ost_sync to node 192.168.201.133@tcp failed: rc = -107 Aug 15 06:24:19 oleg133-client kernel: LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. Aug 15 06:24:19 oleg133-client kernel: Lustre: 2376:0:(llite_lib.c:4340:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.201.133@tcp:/lustre/fid: [0x200000404:0x35:0x0]// may get corrupted (rc -108) Aug 15 06:24:21 oleg133-client sshd[34912]: Accepted publickey for root from 192.168.201.33 port 39398 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:21 oleg133-client systemd-logind[1034]: New session 175 of user root. Aug 15 06:24:21 oleg133-client systemd[1]: Started Session 175 of user root. Aug 15 06:24:21 oleg133-client sshd[34912]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:22 oleg133-client sshd[34925]: Received disconnect from 192.168.201.33 port 39398:11: disconnected by user Aug 15 06:24:22 oleg133-client sshd[34925]: Disconnected from user root 192.168.201.33 port 39398 Aug 15 06:24:22 oleg133-client sshd[34912]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:22 oleg133-client systemd[1]: session-175.scope: Succeeded. Aug 15 06:24:22 oleg133-client systemd-logind[1034]: Session 175 logged out. Waiting for processes to exit. Aug 15 06:24:22 oleg133-client systemd-logind[1034]: Removed session 175. Aug 15 06:24:23 oleg133-client sshd[35050]: Accepted publickey for root from 192.168.201.33 port 39408 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:23 oleg133-client systemd[1]: Started Session 176 of user root. Aug 15 06:24:23 oleg133-client systemd-logind[1034]: New session 176 of user root. Aug 15 06:24:23 oleg133-client sshd[35050]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:23 oleg133-client sshd[35063]: Received disconnect from 192.168.201.33 port 39408:11: disconnected by user Aug 15 06:24:23 oleg133-client sshd[35063]: Disconnected from user root 192.168.201.33 port 39408 Aug 15 06:24:23 oleg133-client sshd[35050]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:23 oleg133-client systemd[1]: session-176.scope: Succeeded. Aug 15 06:24:23 oleg133-client systemd-logind[1034]: Session 176 logged out. Waiting for processes to exit. Aug 15 06:24:23 oleg133-client systemd-logind[1034]: Removed session 176. Aug 15 06:24:25 oleg133-client sshd[35194]: Accepted publickey for root from 192.168.201.33 port 39420 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:25 oleg133-client systemd-logind[1034]: New session 177 of user root. Aug 15 06:24:25 oleg133-client systemd[1]: Started Session 177 of user root. Aug 15 06:24:25 oleg133-client sshd[35194]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:25 oleg133-client sshd[35207]: Received disconnect from 192.168.201.33 port 39420:11: disconnected by user Aug 15 06:24:25 oleg133-client sshd[35207]: Disconnected from user root 192.168.201.33 port 39420 Aug 15 06:24:25 oleg133-client sshd[35194]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:26 oleg133-client systemd[1]: session-177.scope: Succeeded. Aug 15 06:24:26 oleg133-client systemd-logind[1034]: Session 177 logged out. Waiting for processes to exit. Aug 15 06:24:26 oleg133-client systemd-logind[1034]: Removed session 177. Aug 15 06:24:27 oleg133-client sshd[35348]: Accepted publickey for root from 192.168.201.33 port 58740 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:27 oleg133-client systemd-logind[1034]: New session 178 of user root. Aug 15 06:24:27 oleg133-client systemd[1]: Started Session 178 of user root. Aug 15 06:24:27 oleg133-client sshd[35348]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:27 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 26a: evict dead exports =========== 06:24:26 (1786789466) Aug 15 06:24:27 oleg133-client sshd[35361]: Received disconnect from 192.168.201.33 port 58740:11: disconnected by user Aug 15 06:24:27 oleg133-client sshd[35361]: Disconnected from user root 192.168.201.33 port 58740 Aug 15 06:24:27 oleg133-client sshd[35348]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:27 oleg133-client systemd[1]: session-178.scope: Succeeded. Aug 15 06:24:27 oleg133-client systemd-logind[1034]: Session 178 logged out. Waiting for processes to exit. Aug 15 06:24:27 oleg133-client systemd-logind[1034]: Removed session 178. Aug 15 06:24:29 oleg133-client sshd[35577]: Accepted publickey for root from 192.168.201.33 port 58750 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:29 oleg133-client systemd-logind[1034]: New session 179 of user root. Aug 15 06:24:29 oleg133-client systemd[1]: Started Session 179 of user root. Aug 15 06:24:29 oleg133-client sshd[35577]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:30 oleg133-client kernel: Lustre: DEBUG MARKER: SKIP: recovery-small test_26a msg and ost1 are at the same node Aug 15 06:24:30 oleg133-client sshd[35590]: Received disconnect from 192.168.201.33 port 58750:11: disconnected by user Aug 15 06:24:30 oleg133-client sshd[35590]: Disconnected from user root 192.168.201.33 port 58750 Aug 15 06:24:30 oleg133-client sshd[35577]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:30 oleg133-client systemd[1]: session-179.scope: Succeeded. Aug 15 06:24:30 oleg133-client systemd-logind[1034]: Session 179 logged out. Waiting for processes to exit. Aug 15 06:24:30 oleg133-client systemd-logind[1034]: Removed session 179. Aug 15 06:24:31 oleg133-client sshd[35730]: Accepted publickey for root from 192.168.201.33 port 58756 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:31 oleg133-client systemd-logind[1034]: New session 180 of user root. Aug 15 06:24:31 oleg133-client systemd[1]: Started Session 180 of user root. Aug 15 06:24:31 oleg133-client sshd[35730]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:32 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 26b: evict dead exports =========== 06:24:30 (1786789470) Aug 15 06:24:32 oleg133-client sshd[35743]: Received disconnect from 192.168.201.33 port 58756:11: disconnected by user Aug 15 06:24:32 oleg133-client sshd[35743]: Disconnected from user root 192.168.201.33 port 58756 Aug 15 06:24:32 oleg133-client sshd[35730]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:32 oleg133-client systemd[1]: session-180.scope: Succeeded. Aug 15 06:24:32 oleg133-client systemd-logind[1034]: Session 180 logged out. Waiting for processes to exit. Aug 15 06:24:32 oleg133-client systemd-logind[1034]: Removed session 180. Aug 15 06:24:33 oleg133-client sshd[35899]: Accepted publickey for root from 192.168.201.33 port 58768 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:33 oleg133-client systemd-logind[1034]: New session 181 of user root. Aug 15 06:24:33 oleg133-client systemd[1]: Started Session 181 of user root. Aug 15 06:24:33 oleg133-client sshd[35899]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:34 oleg133-client kernel: Lustre: DEBUG MARKER: SKIP: recovery-small test_26b msg and ost1 are at the same node Aug 15 06:24:34 oleg133-client sshd[35912]: Received disconnect from 192.168.201.33 port 58768:11: disconnected by user Aug 15 06:24:34 oleg133-client sshd[35912]: Disconnected from user root 192.168.201.33 port 58768 Aug 15 06:24:34 oleg133-client sshd[35899]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:34 oleg133-client systemd[1]: session-181.scope: Succeeded. Aug 15 06:24:34 oleg133-client systemd-logind[1034]: Session 181 logged out. Waiting for processes to exit. Aug 15 06:24:34 oleg133-client systemd-logind[1034]: Removed session 181. Aug 15 06:24:35 oleg133-client sshd[36051]: Accepted publickey for root from 192.168.201.33 port 36484 ssh2: RSA SHA256:FFZvFiaDhMvadvf0h6nTF/EV7gsGQZ3oCETk+6eowzU Aug 15 06:24:35 oleg133-client systemd-logind[1034]: New session 182 of user root. Aug 15 06:24:35 oleg133-client systemd[1]: Started Session 182 of user root. Aug 15 06:24:35 oleg133-client sshd[36051]: pam_unix(sshd:session): session opened for user root by (uid=0) Aug 15 06:24:36 oleg133-client kernel: Lustre: DEBUG MARKER: == recovery-small test 27: fail LOV while using OSC's ==== 06:24:34 (1786789474) Aug 15 06:24:36 oleg133-client sshd[36064]: Received disconnect from 192.168.201.33 port 36484:11: disconnected by user Aug 15 06:24:36 oleg133-client sshd[36064]: Disconnected from user root 192.168.201.33 port 36484 Aug 15 06:24:36 oleg133-client sshd[36051]: pam_unix(sshd:session): session closed for user root Aug 15 06:24:36 oleg133-client systemd[1]: session-182.scope: Succeeded. Aug 15 06:24:36 oleg133-client systemd-logind[1034]: Session 182 logged out. Waiting for processes to exit. Aug 15 06:24:36 oleg133-client systemd-logind[1034]: Removed session 182. Aug 15 06:24:41 oleg133-client kernel: LustreError: lustre-MDT0000-mdc-ffff993744fd2000: operation mds_reint to node 192.168.201.133@tcp failed: rc = -19 Aug 15 06:24:41 oleg133-client kernel: LustreError: Skipped 1 previous similar message Aug 15 06:25:02 oleg133-client kernel: LustreError: MGC192.168.201.133@tcp: Connection to MGS (at 192.168.201.133@tcp) was lost; in progress operations using this service will fail Aug 15 06:25:02 oleg133-client kernel: Lustre: Evicted from MGS (at 192.168.201.133@tcp) after server handle changed from 0x708664883ac6f82e to 0x708664883ac711bf Aug 15 06:25:04 oleg133-client kernel: Lustre: 15962:0:(mgc_request.c:1899:mgc_process_log()) MGC192.168.201.133@tcp: IR log lustre-cliir failed, not fatal: rc = -2 Aug 15 06:26:54 oleg133-client kernel: LustreError: lustre-MDT0000-mdc-ffff993744fd2000: operation mds_reint to node 192.168.201.133@tcp failed: rc = -19 Aug 15 06:27:15 oleg133-client kernel: LustreError: MGC192.168.201.133@tcp: Connection to MGS (at 192.168.201.133@tcp) was lost; in progress operations using this service will fail Aug 15 06:27:15 oleg133-client kernel: Lustre: Evicted from MGS (at 192.168.201.133@tcp) after server handle changed from 0x708664883ac711bf to 0x708664883ac8a9b7 Aug 15 06:27:17 oleg133-client kernel: Lustre: 15962:0:(mgc_request.c:1899:mgc_process_log()) MGC192.168.201.133@tcp: IR log lustre-cliir failed, not fatal: rc = -2 Aug 15 06:27:34 oleg133-client kernel: BUG: spinlock recursion on CPU#1, ptlrpcd_rcv/2372 Aug 15 06:27:34 oleg133-client kernel: lock: 0xffff99374626eae0, .magic: dead4ead, .owner: ptlrpcd_rcv/2372, .owner_cpu: 1 Aug 15 06:27:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:27:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:27:34 oleg133-client kernel: Call Trace: Aug 15 06:27:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:27:34 oleg133-client kernel: ? spin_dump.cold.3+0x32/0x65 Aug 15 06:27:34 oleg133-client kernel: ? do_raw_spin_lock+0x10f/0x190 Aug 15 06:27:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:27:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:27:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:27:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:27:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:27:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:27:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:27:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:27:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:27:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:27:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:27:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:27:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:27:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:27:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:27:55 oleg133-client kernel: LustreError: MGC192.168.201.133@tcp: Connection to MGS (at 192.168.201.133@tcp) was lost; in progress operations using this service will fail Aug 15 06:28:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:28:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:28:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:28:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:28:00 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:28:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:28:00 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:28:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:28:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:28:00 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 Aug 15 06:28:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:28:00 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:28:00 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:28:00 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:28:00 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:28:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:28:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:28:00 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:28:00 oleg133-client kernel: Call Trace: Aug 15 06:28:00 oleg133-client kernel: Aug 15 06:28:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:28:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:28:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:28:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:28:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:28:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:28:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:28:00 oleg133-client kernel: Aug 15 06:28:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:28:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:28:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:28:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:28:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:28:00 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:28:00 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:28:00 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:28:00 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:28:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:28:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:28:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:28:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:28:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:28:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:28:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:28:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:28:00 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:28:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:28:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:28:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:28:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:28:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:28:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:28:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:28:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:28:00 oleg133-client kernel: Call Trace: Aug 15 06:28:00 oleg133-client kernel: Aug 15 06:28:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:28:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:28:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:28:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:28:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:28:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:28:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:28:00 oleg133-client kernel: Aug 15 06:28:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:28:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:28:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:28:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:28:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:28:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:28:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:28:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:28:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:28:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:28:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:28:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:28:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:28:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:28:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:28:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:28:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:28:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:28:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:28:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:28:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:28:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:28:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:28:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:28:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:28:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:28:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:28:01 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:28:01 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:28:01 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:28:01 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:28:01 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:28:01 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:28:01 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:28:01 oleg133-client kernel: RDX: 0000000000002be3 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:28:01 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:28:01 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:28:01 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:28:01 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:28:01 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:28:01 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:28:01 oleg133-client kernel: Call Trace: Aug 15 06:28:01 oleg133-client kernel: Aug 15 06:28:01 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:28:01 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:28:01 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:28:01 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:28:01 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:28:01 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:28:01 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:28:01 oleg133-client kernel: Aug 15 06:28:01 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:28:01 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:28:01 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:28:01 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:28:01 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:28:01 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:28:01 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:28:01 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:28:01 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:28:01 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:28:01 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:28:01 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:28:01 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:28:01 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:28:01 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:28:01 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:28:01 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:28:01 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:28:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:28:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:28:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:28:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:28:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:28:28 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:28:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:28:28 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:28:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:28:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:28:28 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:28:28 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:28:28 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:28:28 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:28:28 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:28:28 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:28:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:28:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:28:28 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:28:28 oleg133-client kernel: Call Trace: Aug 15 06:28:28 oleg133-client kernel: Aug 15 06:28:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:28:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:28:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:28:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:28:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:28:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:28:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:28:28 oleg133-client kernel: Aug 15 06:28:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:28:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:28:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:28:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:28:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:28:28 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:28:28 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:28:28 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:28:28 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:28:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:28:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:28:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:28:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:28:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:28:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:28:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:28:28 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:28:28 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:28:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:28:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:28:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:28:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:28:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:28:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:28:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:28:28 oleg133-client kernel: Call Trace: Aug 15 06:28:28 oleg133-client kernel: Aug 15 06:28:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:28:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:28:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:28:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:28:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:28:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:28:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:28:28 oleg133-client kernel: Aug 15 06:28:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:28:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:28:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:28:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:28:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:28:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:28:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:28:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:28:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:28:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:28:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:28:28 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:28:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:28:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:28:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:28:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:28:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:28:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:28:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:28:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:28:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:28:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:28:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:28:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:28:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:28:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:28:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:28:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:28:29 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:28:29 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:28:29 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:28:29 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:28:29 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:28:29 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:28:29 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:28:29 oleg133-client kernel: RDX: 000000000000451d RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:28:29 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:28:29 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:28:29 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:28:29 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:28:29 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:28:29 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:28:29 oleg133-client kernel: Call Trace: Aug 15 06:28:29 oleg133-client kernel: Aug 15 06:28:29 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:28:29 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:28:29 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:28:29 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:28:29 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:28:29 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:28:29 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:28:29 oleg133-client kernel: Aug 15 06:28:29 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:28:29 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:28:29 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:28:29 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:28:29 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:28:29 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:28:29 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:28:29 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:28:29 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:28:29 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:28:29 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:28:29 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:28:29 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:28:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:28:29 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:28:29 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:28:29 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:28:29 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:28:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:28:34 oleg133-client kernel: rcu: #0111-....: (26356 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=8957 Aug 15 06:28:34 oleg133-client kernel: #011(t=60005 jiffies g=278549 q=313) Aug 15 06:28:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:28:34 oleg133-client kernel: NMI backtrace for cpu 0 Aug 15 06:28:34 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:28:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:28:34 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:28:34 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:28:34 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 Aug 15 06:28:34 oleg133-client kernel: RAX: 0000000000000291 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:28:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:28:34 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:28:34 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:28:34 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:28:34 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:28:34 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:28:34 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:28:34 oleg133-client kernel: Call Trace: Aug 15 06:28:34 oleg133-client kernel: Aug 15 06:28:34 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:28:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x98/0xa2 Aug 15 06:28:34 oleg133-client kernel: ? nmi_cpu_backtrace_handler+0x15/0x40 Aug 15 06:28:34 oleg133-client kernel: ? nmi_handle+0xbd/0x210 Aug 15 06:28:34 oleg133-client kernel: ? default_do_nmi+0x4b/0x1b0 Aug 15 06:28:34 oleg133-client kernel: ? do_nmi+0x280/0x380 Aug 15 06:28:34 oleg133-client kernel: ? end_repeat_nmi+0x16/0x1f Aug 15 06:28:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:28:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:28:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:28:34 oleg133-client kernel: Aug 15 06:28:34 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:28:34 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:28:34 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:28:34 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:28:34 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:28:34 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:28:34 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:28:34 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:28:34 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:28:34 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:28:34 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:28:34 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:28:34 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:28:34 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:28:34 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:28:34 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:28:34 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:28:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:28:34 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:28:34 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:28:34 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:28:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:28:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:28:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:28:34 oleg133-client kernel: Call Trace: Aug 15 06:28:34 oleg133-client kernel: Aug 15 06:28:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:28:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:28:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:28:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:28:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:28:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:28:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:28:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:28:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:28:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:28:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:28:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:28:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:28:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:28:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:28:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:28:34 oleg133-client kernel: Aug 15 06:28:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:28:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:28:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:28:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:28:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:28:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:28:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:28:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:28:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:28:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:28:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:28:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:28:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:28:56 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:28:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:28:56 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:28:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:28:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:28:56 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 Aug 15 06:28:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:28:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:28:56 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:28:56 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:28:56 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:28:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:28:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:28:56 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:28:56 oleg133-client kernel: Call Trace: Aug 15 06:28:56 oleg133-client kernel: Aug 15 06:28:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:28:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:28:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:28:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:28:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:28:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:28:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:28:56 oleg133-client kernel: Aug 15 06:28:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:28:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:28:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:28:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:28:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:28:56 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:28:56 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:28:56 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:28:56 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:28:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:28:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:28:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:29:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:29:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:29:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:29:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:29:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:29:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:29:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:29:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:29:00 oleg133-client kernel: RAX: 0000000000006474 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:29:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:29:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:29:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:29:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:29:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:29:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:29:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:29:00 oleg133-client kernel: Call Trace: Aug 15 06:29:00 oleg133-client kernel: Aug 15 06:29:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:29:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:29:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:29:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:29:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:29:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:29:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:29:00 oleg133-client kernel: Aug 15 06:29:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:29:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:29:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:29:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:29:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: Modules linked in: Aug 15 06:29:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:29:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) Aug 15 06:29:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:29:00 oleg133-client kernel: fid(O) Aug 15 06:29:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:29:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:29:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:29:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:29:00 oleg133-client kernel: fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:29:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:29:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:29:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:29:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:29:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:29:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:29:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:29:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:29:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:29:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:29:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:29:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:29:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:29:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:29:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:29:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:29:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:29:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:29:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:29:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:29:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:29:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:29:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:29:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:29:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:29:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:29:00 oleg133-client kernel: Call Trace: Aug 15 06:29:00 oleg133-client kernel: Aug 15 06:29:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:29:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:29:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:29:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:29:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:29:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:29:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:29:00 oleg133-client kernel: Aug 15 06:29:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:29:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:29:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:29:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:29:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:29:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:29:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:29:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:29:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:29:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:29:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:29:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:29:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:29:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:29:24 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:29:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:29:24 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:29:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:29:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:29:24 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:29:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:29:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:29:24 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:29:24 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:29:24 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:29:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:29:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:29:24 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:29:24 oleg133-client kernel: Call Trace: Aug 15 06:29:24 oleg133-client kernel: Aug 15 06:29:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:29:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:29:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:29:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:29:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:29:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:29:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:29:24 oleg133-client kernel: Aug 15 06:29:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:29:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:29:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:29:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:29:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:29:24 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:29:24 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:29:24 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:29:24 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:29:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:29:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:29:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:29:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:29:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:29:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) Aug 15 06:29:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:29:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:29:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:29:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:29:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:29:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:29:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:29:28 oleg133-client kernel: RDX: 0000000000006b0e RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:29:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:29:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:29:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:29:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:29:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:29:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:29:28 oleg133-client kernel: Call Trace: Aug 15 06:29:28 oleg133-client kernel: Aug 15 06:29:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:29:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:29:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:29:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:29:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:29:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:29:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:29:28 oleg133-client kernel: Aug 15 06:29:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:29:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:29:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:29:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:29:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:29:28 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:29:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:29:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:29:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:29:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:29:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:29:28 oleg133-client kernel: RAX: 0000000000007f89 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:29:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:29:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:29:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:29:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:29:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:29:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:29:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:29:28 oleg133-client kernel: Call Trace: Aug 15 06:29:28 oleg133-client kernel: Aug 15 06:29:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:29:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:29:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:29:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:29:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:29:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:29:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:29:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:29:28 oleg133-client kernel: Aug 15 06:29:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:29:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:29:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:29:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:29:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:29:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:29:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:29:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:29:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:29:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:29:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:29:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:29:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:29:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:29:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:29:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:29:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:29:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:29:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:29:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:29:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:29:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:29:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:29:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:29:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:29:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:29:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:29:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:29:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:29:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:29:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:29:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:29:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:29:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:29:29 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:29:29 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:29:29 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:29:29 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:29:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:29:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:29:52 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:29:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:29:52 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:29:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:29:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:29:52 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:29:52 oleg133-client kernel: RAX: 0000000000007dfd RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:29:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:29:52 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:29:52 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:29:52 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:29:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:29:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:29:52 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:29:52 oleg133-client kernel: Call Trace: Aug 15 06:29:52 oleg133-client kernel: Aug 15 06:29:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:29:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:29:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:29:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:29:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:29:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:29:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:29:52 oleg133-client kernel: Aug 15 06:29:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:29:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:29:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:29:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:29:52 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:29:52 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:29:52 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:29:52 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:29:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:29:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:29:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:29:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:29:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) Aug 15 06:29:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:29:56 oleg133-client kernel: mgs(O) Aug 15 06:29:56 oleg133-client kernel: Modules linked in: Aug 15 06:29:56 oleg133-client kernel: lquota(O) Aug 15 06:29:56 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:29:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:29:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:29:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:29:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:29:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:29:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:29:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:29:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:29:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:29:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:29:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:29:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:29:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:29:56 oleg133-client kernel: Call Trace: Aug 15 06:29:56 oleg133-client kernel: Aug 15 06:29:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:29:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:29:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:29:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:29:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:29:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:29:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:29:56 oleg133-client kernel: Aug 15 06:29:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:29:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:29:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:29:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:29:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:29:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:29:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:29:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:29:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:29:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:29:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:29:56 oleg133-client kernel: lfsck(O) Aug 15 06:29:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:29:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:29:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:29:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:29:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:29:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:29:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:29:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:29:57 oleg133-client kernel: obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:29:57 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:29:57 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:29:57 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:29:57 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:29:57 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:29:57 oleg133-client kernel: RAX: 0000000000005757 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:29:57 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:29:57 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:29:57 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:29:57 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:29:57 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:29:57 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:29:57 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:29:57 oleg133-client kernel: Call Trace: Aug 15 06:29:57 oleg133-client kernel: Aug 15 06:29:57 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:29:57 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:29:57 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:29:57 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:29:57 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:29:57 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:29:57 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:29:57 oleg133-client kernel: Aug 15 06:29:57 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:29:57 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:29:57 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:29:57 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:29:57 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:29:57 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:29:57 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:29:57 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:29:57 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:29:57 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:29:57 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:29:57 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:29:57 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:29:57 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:29:57 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:29:57 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:29:57 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:29:57 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:29:57 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:29:57 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:29:57 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:29:57 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:29:57 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:29:57 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:29:57 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:29:57 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:30:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:30:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:30:20 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:30:20 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:30:20 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:30:20 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:30:20 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:30:20 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:30:20 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:30:20 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:30:20 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:30:20 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:30:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:30:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:30:20 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:30:20 oleg133-client kernel: Call Trace: Aug 15 06:30:20 oleg133-client kernel: Aug 15 06:30:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:30:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:30:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:30:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:30:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:30:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:30:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:30:20 oleg133-client kernel: Aug 15 06:30:20 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:30:20 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:30:20 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:30:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:30:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:30:20 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:30:20 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:30:20 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:30:20 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:30:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:30:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:30:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:30:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:30:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:30:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:30:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:30:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:30:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:30:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:30:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:30:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:30:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:30:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:30:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:30:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:30:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:30:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:30:24 oleg133-client kernel: Call Trace: Aug 15 06:30:24 oleg133-client kernel: Aug 15 06:30:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:30:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:30:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:30:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:30:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:30:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:30:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:30:24 oleg133-client kernel: Aug 15 06:30:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:30:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:30:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:30:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:30:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:30:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:30:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:30:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:30:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:30:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:30:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:30:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:30:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:30:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:30:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:30:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:30:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:30:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:30:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:30:24 oleg133-client kernel: Call Trace: Aug 15 06:30:24 oleg133-client kernel: Aug 15 06:30:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:30:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:30:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:30:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:30:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:30:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:30:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:30:24 oleg133-client kernel: Aug 15 06:30:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:30:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:30:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:30:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:30:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:30:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:30:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:30:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:30:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:30:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:30:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:30:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:30:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:30:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:30:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:30:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:30:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:30:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:30:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:30:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:30:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:30:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:30:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:30:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:30:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:30:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:30:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:30:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:30:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:30:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:30:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:30:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:30:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:30:25 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:30:25 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:30:25 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:30:25 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:30:25 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:30:25 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:30:25 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:30:48 oleg133-client kernel: INFO: task khugepaged:48 blocked for more than 120 seconds. Aug 15 06:30:48 oleg133-client kernel: Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:48 oleg133-client kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Aug 15 06:30:48 oleg133-client kernel: task:khugepaged state:D stack:0 pid:48 ppid:2 flags:0x80004000 Aug 15 06:30:48 oleg133-client kernel: Call Trace: Aug 15 06:30:48 oleg133-client kernel: __schedule+0x351/0xcb0 Aug 15 06:30:48 oleg133-client kernel: ? wait_for_completion+0xae/0x1e0 Aug 15 06:30:48 oleg133-client kernel: schedule+0xc0/0x180 Aug 15 06:30:48 oleg133-client kernel: schedule_timeout+0x126/0x190 Aug 15 06:30:48 oleg133-client kernel: ? try_to_wake_up+0x80/0x710 Aug 15 06:30:48 oleg133-client kernel: ? __prepare_to_swait+0x5b/0x90 Aug 15 06:30:48 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:30:48 oleg133-client kernel: wait_for_completion+0xf0/0x1e0 Aug 15 06:30:48 oleg133-client kernel: flush_work+0x1f3/0x390 Aug 15 06:30:48 oleg133-client kernel: ? flush_workqueue_prep_pwqs+0x200/0x200 Aug 15 06:30:48 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:30:48 oleg133-client kernel: ? __pagevec_release+0x50/0x50 Aug 15 06:30:48 oleg133-client kernel: lru_add_drain_all+0x1d4/0x280 Aug 15 06:30:48 oleg133-client kernel: khugepaged+0xa2/0x16f0 Aug 15 06:30:48 oleg133-client kernel: ? woken_wake_function+0x30/0x30 Aug 15 06:30:48 oleg133-client kernel: ? collapse_pte_mapped_thp+0x8a0/0x8a0 Aug 15 06:30:48 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:30:48 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:30:48 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:30:48 oleg133-client kernel: INFO: task writemany:36103 blocked for more than 120 seconds. Aug 15 06:30:48 oleg133-client kernel: Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:48 oleg133-client kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Aug 15 06:30:48 oleg133-client kernel: task:writemany state:D stack:0 pid:36103 ppid:36100 flags:0x80000004 Aug 15 06:30:48 oleg133-client kernel: Call Trace: Aug 15 06:30:48 oleg133-client kernel: __schedule+0x351/0xcb0 Aug 15 06:30:48 oleg133-client kernel: schedule+0xc0/0x180 Aug 15 06:30:48 oleg133-client kernel: schedule_preempt_disabled+0x21/0x40 Aug 15 06:30:48 oleg133-client kernel: rwsem_down_write_slowpath+0x5d7/0xa40 Aug 15 06:30:48 oleg133-client kernel: ? lprocfs_counter_add+0x15b/0x210 [obdclass] Aug 15 06:30:48 oleg133-client kernel: down_write+0x80/0xd0 Aug 15 06:30:48 oleg133-client kernel: do_last+0x2eb/0xfc0 Aug 15 06:30:48 oleg133-client kernel: ? nd_jump_root+0xe5/0x160 Aug 15 06:30:48 oleg133-client kernel: ? path_init+0x437/0x520 Aug 15 06:30:48 oleg133-client kernel: path_openat+0xf7/0x500 Aug 15 06:30:48 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:30:48 oleg133-client kernel: do_filp_open+0x99/0x140 Aug 15 06:30:48 oleg133-client kernel: ? getname_flags+0x6e/0x330 Aug 15 06:30:48 oleg133-client kernel: ? __check_object_size+0xff/0x256 Aug 15 06:30:48 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:30:48 oleg133-client kernel: ? _raw_spin_unlock+0x12/0x30 Aug 15 06:30:48 oleg133-client kernel: do_sys_openat2+0x2b4/0x410 Aug 15 06:30:48 oleg133-client kernel: do_sys_open+0x73/0xa0 Aug 15 06:30:48 oleg133-client kernel: __x64_sys_openat+0x24/0x30 Aug 15 06:30:48 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:30:48 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:30:48 oleg133-client kernel: RIP: 0033:0x7fe02c20e332 Aug 15 06:30:48 oleg133-client kernel: Code: Unable to access opcode bytes at RIP 0x7fe02c20e308. Aug 15 06:30:48 oleg133-client kernel: RSP: 002b:00007fff45ac4b90 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 Aug 15 06:30:48 oleg133-client kernel: RAX: ffffffffffffffda RBX: 00000000000191d8 RCX: 00007fe02c20e332 Aug 15 06:30:48 oleg133-client kernel: RDX: 0000000000000042 RSI: 00007fff45ac4c60 RDI: 00000000ffffff9c Aug 15 06:30:48 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:30:48 oleg133-client kernel: R10: 00000000000001b6 R11: 0000000000000246 R12: 00007fff45ac4c60 Aug 15 06:30:48 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000191 R15: 0000000000000000 Aug 15 06:30:48 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:30:48 oleg133-client kernel: INFO: task writemany:36104 blocked for more than 120 seconds. Aug 15 06:30:48 oleg133-client kernel: Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:48 oleg133-client kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Aug 15 06:30:48 oleg133-client kernel: task:writemany state:D stack:0 pid:36104 ppid:36100 flags:0x80000004 Aug 15 06:30:48 oleg133-client kernel: Call Trace: Aug 15 06:30:48 oleg133-client kernel: __schedule+0x351/0xcb0 Aug 15 06:30:48 oleg133-client kernel: schedule+0xc0/0x180 Aug 15 06:30:48 oleg133-client kernel: schedule_preempt_disabled+0x21/0x40 Aug 15 06:30:48 oleg133-client kernel: rwsem_down_write_slowpath+0x5d7/0xa40 Aug 15 06:30:48 oleg133-client kernel: ? lprocfs_counter_add+0x15b/0x210 [obdclass] Aug 15 06:30:48 oleg133-client kernel: down_write+0x80/0xd0 Aug 15 06:30:48 oleg133-client kernel: do_last+0x2eb/0xfc0 Aug 15 06:30:48 oleg133-client kernel: ? nd_jump_root+0xe5/0x160 Aug 15 06:30:48 oleg133-client kernel: ? path_init+0x437/0x520 Aug 15 06:30:48 oleg133-client kernel: path_openat+0xf7/0x500 Aug 15 06:30:48 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:30:48 oleg133-client kernel: do_filp_open+0x99/0x140 Aug 15 06:30:48 oleg133-client kernel: ? getname_flags+0x6e/0x330 Aug 15 06:30:48 oleg133-client kernel: ? __check_object_size+0xff/0x256 Aug 15 06:30:48 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:30:48 oleg133-client kernel: ? _raw_spin_unlock+0x12/0x30 Aug 15 06:30:48 oleg133-client kernel: do_sys_openat2+0x2b4/0x410 Aug 15 06:30:48 oleg133-client kernel: do_sys_open+0x73/0xa0 Aug 15 06:30:48 oleg133-client kernel: __x64_sys_openat+0x24/0x30 Aug 15 06:30:48 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:30:48 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:30:48 oleg133-client kernel: RIP: 0033:0x7fe02c20e332 Aug 15 06:30:48 oleg133-client kernel: Code: Unable to access opcode bytes at RIP 0x7fe02c20e308. Aug 15 06:30:48 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:30:48 oleg133-client kernel: RSP: 002b:00007fff45ac4b90 EFLAGS: 00000246 Aug 15 06:30:48 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) Aug 15 06:30:48 oleg133-client kernel: ORIG_RAX: 0000000000000101 Aug 15 06:30:48 oleg133-client kernel: RAX: ffffffffffffffda RBX: 00000000000f0290 RCX: 00007fe02c20e332 Aug 15 06:30:48 oleg133-client kernel: RDX: 0000000000000042 RSI: 00007fff45ac4c60 RDI: 00000000ffffff9c Aug 15 06:30:48 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:30:48 oleg133-client kernel: R10: 00000000000001b6 R11: 0000000000000246 R12: 00007fff45ac4c60 Aug 15 06:30:48 oleg133-client kernel: R13: 00000000ffffffff R14: 000000000000015f R15: 0000000000000000 Aug 15 06:30:48 oleg133-client kernel: INFO: task writemany:36105 blocked for more than 120 seconds. Aug 15 06:30:48 oleg133-client kernel: obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:30:48 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:48 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:30:48 oleg133-client kernel: Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:48 oleg133-client kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Aug 15 06:30:48 oleg133-client kernel: task:writemany state:D stack:0 pid:36105 ppid:36100 flags:0x80000004 Aug 15 06:30:48 oleg133-client kernel: Call Trace: Aug 15 06:30:48 oleg133-client kernel: __schedule+0x351/0xcb0 Aug 15 06:30:48 oleg133-client kernel: schedule+0xc0/0x180 Aug 15 06:30:48 oleg133-client kernel: schedule_preempt_disabled+0x21/0x40 Aug 15 06:30:48 oleg133-client kernel: rwsem_down_write_slowpath+0x5d7/0xa40 Aug 15 06:30:48 oleg133-client kernel: ? __might_sleep+0x59/0xc0 Aug 15 06:30:48 oleg133-client kernel: down_write+0x80/0xd0 Aug 15 06:30:48 oleg133-client kernel: do_unlinkat+0x184/0x460 Aug 15 06:30:48 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:30:48 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:30:48 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:30:48 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:30:48 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:30:48 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:30:48 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:30:48 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:30:48 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:30:48 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:30:48 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:30:48 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:30:48 oleg133-client kernel: Call Trace: Aug 15 06:30:48 oleg133-client kernel: Aug 15 06:30:48 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:30:48 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:30:48 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:30:48 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:30:48 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:30:48 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:30:48 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:30:48 oleg133-client kernel: Aug 15 06:30:48 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:30:48 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:30:48 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:30:48 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:30:48 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:30:48 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:30:48 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:30:48 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:30:48 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:30:48 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:30:48 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:30:48 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:30:48 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:30:48 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:30:48 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:30:48 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:30:48 oleg133-client kernel: Code: Unable to access opcode bytes at RIP 0x7fe02c1190e1. Aug 15 06:30:48 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:30:48 oleg133-client kernel: RAX: ffffffffffffffda RBX: 000000000058d508 RCX: 00007fe02c11910b Aug 15 06:30:48 oleg133-client kernel: RDX: 0000000000000038 RSI: 00007fff45ac5060 RDI: 00007fff45ac4c60 Aug 15 06:30:48 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1228 R09: 00007fe02c4b1260 Aug 15 06:30:48 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:30:48 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:30:48 oleg133-client kernel: INFO: task writemany:36106 blocked for more than 120 seconds. Aug 15 06:30:48 oleg133-client kernel: Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:48 oleg133-client kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Aug 15 06:30:48 oleg133-client kernel: task:writemany state:D stack:0 pid:36106 ppid:36100 flags:0x80000004 Aug 15 06:30:48 oleg133-client kernel: Call Trace: Aug 15 06:30:48 oleg133-client kernel: __schedule+0x351/0xcb0 Aug 15 06:30:48 oleg133-client kernel: schedule+0xc0/0x180 Aug 15 06:30:48 oleg133-client kernel: schedule_preempt_disabled+0x21/0x40 Aug 15 06:30:48 oleg133-client kernel: rwsem_down_write_slowpath+0x5d7/0xa40 Aug 15 06:30:48 oleg133-client kernel: ? __might_sleep+0x59/0xc0 Aug 15 06:30:48 oleg133-client kernel: down_write+0x80/0xd0 Aug 15 06:30:48 oleg133-client kernel: do_unlinkat+0x184/0x460 Aug 15 06:30:48 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:30:48 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:30:48 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:30:48 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:30:48 oleg133-client kernel: Code: Unable to access opcode bytes at RIP 0x7fe02c1190e1. Aug 15 06:30:48 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:30:48 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000c97218 RCX: 00007fe02c11910b Aug 15 06:30:48 oleg133-client kernel: RDX: 0000000000000038 RSI: 00007fff45ac5060 RDI: 00007fff45ac4c60 Aug 15 06:30:48 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b122c R09: 00007fe02c4b1260 Aug 15 06:30:48 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:30:48 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:30:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:30:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:30:52 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:30:52 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:30:52 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:30:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:30:52 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:30:52 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:30:52 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:30:52 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:30:52 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:30:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:30:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:30:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:30:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:30:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:30:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:30:52 oleg133-client kernel: Call Trace: Aug 15 06:30:52 oleg133-client kernel: Aug 15 06:30:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:30:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:30:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:30:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:30:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:30:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:30:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:30:52 oleg133-client kernel: Aug 15 06:30:52 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:30:52 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:30:52 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:30:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:30:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:30:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:30:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:30:52 oleg133-client kernel: osp(O) Aug 15 06:30:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: ofd(O) Aug 15 06:30:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:30:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:30:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:30:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:30:52 oleg133-client kernel: lod(O) Aug 15 06:30:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:30:52 oleg133-client kernel: mdt(O) Aug 15 06:30:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:30:52 oleg133-client kernel: mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:30:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:30:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:30:52 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:30:52 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:30:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:30:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:30:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:30:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:30:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:30:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:30:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:30:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:30:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:30:52 oleg133-client kernel: Call Trace: Aug 15 06:30:52 oleg133-client kernel: Aug 15 06:30:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:30:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:30:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:30:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:30:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:30:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:30:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:30:52 oleg133-client kernel: Aug 15 06:30:52 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:30:52 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:30:52 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:30:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:30:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:30:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:30:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:30:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:30:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:30:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:30:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:30:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:30:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:30:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:30:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:30:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:30:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:30:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:30:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:30:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:30:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:30:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:30:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:31:16 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:31:16 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:31:16 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:31:16 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:31:16 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:31:16 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:31:16 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:31:16 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:31:16 oleg133-client kernel: RAX: 0000000000004b1a RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:31:16 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:31:16 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:31:16 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:31:16 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:31:16 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:31:16 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:31:16 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:31:16 oleg133-client kernel: Call Trace: Aug 15 06:31:16 oleg133-client kernel: Aug 15 06:31:16 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:31:16 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:31:16 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:31:16 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:31:16 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:31:16 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:31:16 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:31:16 oleg133-client kernel: Aug 15 06:31:16 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:31:16 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:31:16 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:31:16 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:31:16 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:31:16 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:31:16 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:31:16 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:31:16 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:31:16 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:31:16 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:31:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:31:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:31:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:31:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:31:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:31:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:31:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:31:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:31:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:31:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:31:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:31:20 oleg133-client kernel: RDX: 0000000000001fa6 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:31:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:31:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:31:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:31:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:31:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:31:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:31:20 oleg133-client kernel: Call Trace: Aug 15 06:31:20 oleg133-client kernel: Aug 15 06:31:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:31:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:31:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:31:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:31:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:31:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:31:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:31:20 oleg133-client kernel: Aug 15 06:31:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:31:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:31:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:31:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:31:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:31:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:31:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:31:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:31:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:31:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:31:20 oleg133-client kernel: RAX: 0000000000005670 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:31:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:31:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:31:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:31:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:31:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:31:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:31:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:31:20 oleg133-client kernel: Call Trace: Aug 15 06:31:20 oleg133-client kernel: Aug 15 06:31:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:31:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:31:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:31:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:31:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:31:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:31:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:31:20 oleg133-client kernel: Aug 15 06:31:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:31:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:31:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:31:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:31:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:31:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:31:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:31:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:31:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:31:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:31:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:31:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:31:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:31:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:31:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:31:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:31:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:31:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:31:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:31:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:31:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:31:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:31:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:31:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:31:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:31:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:31:21 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:31:21 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:31:21 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:31:21 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:31:21 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:31:21 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:31:21 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:31:21 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:31:21 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:31:21 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:31:21 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:31:21 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:31:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:31:34 oleg133-client kernel: rcu: #0111-....: (145612 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=43610 Aug 15 06:31:34 oleg133-client kernel: #011(t=240008 jiffies g=278549 q=1254) Aug 15 06:31:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:31:34 oleg133-client kernel: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 Aug 15 06:31:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:31:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:31:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:31:34 oleg133-client kernel: Call Trace: Aug 15 06:31:34 oleg133-client kernel: Aug 15 06:31:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:31:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:31:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:31:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:31:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:31:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:31:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:31:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:31:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:31:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:31:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:31:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:31:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:31:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:31:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:31:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:31:34 oleg133-client kernel: Aug 15 06:31:34 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:31:34 oleg133-client kernel: ? kvm_wait+0xcd/0xe0 Aug 15 06:31:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:31:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:31:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:31:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:31:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:31:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:31:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:31:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:31:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:31:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:31:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:31:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:31:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:31:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:31:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:31:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:31:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:31:44 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:31:44 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:31:44 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:31:44 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:31:44 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:31:44 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:31:44 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:31:44 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:31:44 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:31:44 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:31:44 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:31:44 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:31:44 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:31:44 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:31:44 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:31:44 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:31:44 oleg133-client kernel: Call Trace: Aug 15 06:31:44 oleg133-client kernel: Aug 15 06:31:44 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:31:44 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:31:44 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:31:44 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:31:44 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:31:44 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:31:44 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:31:44 oleg133-client kernel: Aug 15 06:31:44 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:31:44 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:31:44 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:31:44 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:31:44 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:31:44 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:31:44 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:31:44 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:31:44 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:31:44 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:31:44 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:31:44 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:32:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:32:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:32:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:32:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:32:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:32:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:32:00 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 Aug 15 06:32:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:32:00 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:32:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:32:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:32:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:32:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:32:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:32:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:32:00 oleg133-client kernel: Call Trace: Aug 15 06:32:00 oleg133-client kernel: Aug 15 06:32:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:32:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:32:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:32:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:32:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:32:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:32:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:32:00 oleg133-client kernel: Aug 15 06:32:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:32:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:32:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:32:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:32:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:32:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:32:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:32:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:32:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:32:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:32:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:32:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:32:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:32:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:32:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:32:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:32:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:32:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:32:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:32:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:32:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:32:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:32:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:32:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:32:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:32:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:32:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:32:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:32:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:32:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:32:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:32:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:32:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:32:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:32:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:32:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:32:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:32:00 oleg133-client kernel: Call Trace: Aug 15 06:32:00 oleg133-client kernel: Aug 15 06:32:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:32:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:32:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:32:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:32:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:32:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:32:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:32:00 oleg133-client kernel: Aug 15 06:32:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:32:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:32:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:32:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:32:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:32:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:32:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:32:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:32:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:32:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:32:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:32:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:32:12 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:32:12 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:32:12 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:12 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:32:12 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:32:12 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:32:12 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:32:12 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:32:12 oleg133-client kernel: RAX: 00000000000010b2 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:32:12 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:32:12 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:32:12 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:32:12 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:32:12 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:32:12 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:32:12 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:32:12 oleg133-client kernel: Call Trace: Aug 15 06:32:12 oleg133-client kernel: Aug 15 06:32:12 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:32:12 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:32:12 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:32:12 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:32:12 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:32:12 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:32:12 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:32:12 oleg133-client kernel: Aug 15 06:32:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:32:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:32:12 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:32:12 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:32:12 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:32:12 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:32:12 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:32:12 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:32:12 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:32:12 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:32:12 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:32:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:32:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:32:28 oleg133-client kernel: Modules linked in: Aug 15 06:32:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:32:28 oleg133-client kernel: lustre(O) Aug 15 06:32:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:32:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:32:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:32:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:32:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:32:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:32:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:32:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:32:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:32:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:32:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:32:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:32:28 oleg133-client kernel: Call Trace: Aug 15 06:32:28 oleg133-client kernel: Aug 15 06:32:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:32:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:32:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:32:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:32:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:32:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:32:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:32:28 oleg133-client kernel: Aug 15 06:32:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:32:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:32:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:32:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:32:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:32:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:32:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:32:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:32:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:32:28 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:32:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:32:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:32:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:32:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:32:28 oleg133-client kernel: RAX: 00000000000021b4 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:32:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:32:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:32:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:32:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:32:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:32:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:32:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:32:28 oleg133-client kernel: Call Trace: Aug 15 06:32:28 oleg133-client kernel: Aug 15 06:32:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:32:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:32:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:32:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:32:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:32:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:32:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:32:28 oleg133-client kernel: Aug 15 06:32:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:32:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:32:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:32:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:32:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:32:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:32:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:32:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:32:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:32:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:32:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:32:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:32:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:32:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:32:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:32:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:32:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:32:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:32:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:32:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:32:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:32:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:32:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:32:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:32:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:32:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:32:29 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:32:29 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:32:29 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:32:29 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:32:29 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:32:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:32:29 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:32:29 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:32:29 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:32:29 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:32:40 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:32:40 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:32:40 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:40 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:32:40 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:32:40 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:32:40 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:32:40 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:32:40 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:32:40 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:32:40 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:32:40 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:32:40 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:32:40 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:32:40 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:32:40 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:32:40 oleg133-client kernel: Call Trace: Aug 15 06:32:40 oleg133-client kernel: Aug 15 06:32:40 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:32:40 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:32:40 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:32:40 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:32:40 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:32:40 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:32:40 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:32:40 oleg133-client kernel: Aug 15 06:32:40 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:32:40 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:32:40 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:32:40 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:32:40 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:32:40 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:32:40 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:32:40 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:32:40 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:32:40 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:32:40 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:32:40 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:32:51 oleg133-client kernel: INFO: task khugepaged:48 blocked for more than 120 seconds. Aug 15 06:32:51 oleg133-client kernel: Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:51 oleg133-client kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Aug 15 06:32:51 oleg133-client kernel: task:khugepaged state:D stack:0 pid:48 ppid:2 flags:0x80004000 Aug 15 06:32:51 oleg133-client kernel: Call Trace: Aug 15 06:32:51 oleg133-client kernel: __schedule+0x351/0xcb0 Aug 15 06:32:51 oleg133-client kernel: ? wait_for_completion+0xae/0x1e0 Aug 15 06:32:51 oleg133-client kernel: schedule+0xc0/0x180 Aug 15 06:32:51 oleg133-client kernel: schedule_timeout+0x126/0x190 Aug 15 06:32:51 oleg133-client kernel: ? try_to_wake_up+0x80/0x710 Aug 15 06:32:51 oleg133-client kernel: ? __prepare_to_swait+0x5b/0x90 Aug 15 06:32:51 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:32:51 oleg133-client kernel: wait_for_completion+0xf0/0x1e0 Aug 15 06:32:51 oleg133-client kernel: flush_work+0x1f3/0x390 Aug 15 06:32:51 oleg133-client kernel: ? flush_workqueue_prep_pwqs+0x200/0x200 Aug 15 06:32:51 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:32:51 oleg133-client kernel: ? __pagevec_release+0x50/0x50 Aug 15 06:32:51 oleg133-client kernel: lru_add_drain_all+0x1d4/0x280 Aug 15 06:32:51 oleg133-client kernel: khugepaged+0xa2/0x16f0 Aug 15 06:32:51 oleg133-client kernel: ? woken_wake_function+0x30/0x30 Aug 15 06:32:51 oleg133-client kernel: ? collapse_pte_mapped_thp+0x8a0/0x8a0 Aug 15 06:32:51 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:32:51 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:32:51 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:32:51 oleg133-client kernel: INFO: task writemany:36103 blocked for more than 120 seconds. Aug 15 06:32:51 oleg133-client kernel: Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:51 oleg133-client kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Aug 15 06:32:51 oleg133-client kernel: task:writemany state:D stack:0 pid:36103 ppid:36100 flags:0x80000004 Aug 15 06:32:51 oleg133-client kernel: Call Trace: Aug 15 06:32:51 oleg133-client kernel: __schedule+0x351/0xcb0 Aug 15 06:32:51 oleg133-client kernel: schedule+0xc0/0x180 Aug 15 06:32:51 oleg133-client kernel: schedule_preempt_disabled+0x21/0x40 Aug 15 06:32:51 oleg133-client kernel: rwsem_down_write_slowpath+0x5d7/0xa40 Aug 15 06:32:51 oleg133-client kernel: ? lprocfs_counter_add+0x15b/0x210 [obdclass] Aug 15 06:32:51 oleg133-client kernel: down_write+0x80/0xd0 Aug 15 06:32:51 oleg133-client kernel: do_last+0x2eb/0xfc0 Aug 15 06:32:51 oleg133-client kernel: ? nd_jump_root+0xe5/0x160 Aug 15 06:32:51 oleg133-client kernel: ? path_init+0x437/0x520 Aug 15 06:32:51 oleg133-client kernel: path_openat+0xf7/0x500 Aug 15 06:32:51 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:32:51 oleg133-client kernel: do_filp_open+0x99/0x140 Aug 15 06:32:51 oleg133-client kernel: ? getname_flags+0x6e/0x330 Aug 15 06:32:51 oleg133-client kernel: ? __check_object_size+0xff/0x256 Aug 15 06:32:51 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:32:51 oleg133-client kernel: ? _raw_spin_unlock+0x12/0x30 Aug 15 06:32:51 oleg133-client kernel: do_sys_openat2+0x2b4/0x410 Aug 15 06:32:51 oleg133-client kernel: do_sys_open+0x73/0xa0 Aug 15 06:32:51 oleg133-client kernel: __x64_sys_openat+0x24/0x30 Aug 15 06:32:51 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:32:51 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:32:51 oleg133-client kernel: RIP: 0033:0x7fe02c20e332 Aug 15 06:32:51 oleg133-client kernel: Code: Unable to access opcode bytes at RIP 0x7fe02c20e308. Aug 15 06:32:51 oleg133-client kernel: RSP: 002b:00007fff45ac4b90 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 Aug 15 06:32:51 oleg133-client kernel: RAX: ffffffffffffffda RBX: 00000000000191d8 RCX: 00007fe02c20e332 Aug 15 06:32:51 oleg133-client kernel: RDX: 0000000000000042 RSI: 00007fff45ac4c60 RDI: 00000000ffffff9c Aug 15 06:32:51 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:32:51 oleg133-client kernel: R10: 00000000000001b6 R11: 0000000000000246 R12: 00007fff45ac4c60 Aug 15 06:32:51 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000191 R15: 0000000000000000 Aug 15 06:32:51 oleg133-client kernel: INFO: task writemany:36104 blocked for more than 120 seconds. Aug 15 06:32:51 oleg133-client kernel: Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:51 oleg133-client kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Aug 15 06:32:51 oleg133-client kernel: task:writemany state:D stack:0 pid:36104 ppid:36100 flags:0x80000004 Aug 15 06:32:51 oleg133-client kernel: Call Trace: Aug 15 06:32:51 oleg133-client kernel: __schedule+0x351/0xcb0 Aug 15 06:32:51 oleg133-client kernel: schedule+0xc0/0x180 Aug 15 06:32:51 oleg133-client kernel: schedule_preempt_disabled+0x21/0x40 Aug 15 06:32:51 oleg133-client kernel: rwsem_down_write_slowpath+0x5d7/0xa40 Aug 15 06:32:51 oleg133-client kernel: ? lprocfs_counter_add+0x15b/0x210 [obdclass] Aug 15 06:32:51 oleg133-client kernel: down_write+0x80/0xd0 Aug 15 06:32:51 oleg133-client kernel: do_last+0x2eb/0xfc0 Aug 15 06:32:51 oleg133-client kernel: ? nd_jump_root+0xe5/0x160 Aug 15 06:32:51 oleg133-client kernel: ? path_init+0x437/0x520 Aug 15 06:32:51 oleg133-client kernel: path_openat+0xf7/0x500 Aug 15 06:32:51 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:32:51 oleg133-client kernel: do_filp_open+0x99/0x140 Aug 15 06:32:51 oleg133-client kernel: ? getname_flags+0x6e/0x330 Aug 15 06:32:51 oleg133-client kernel: ? __check_object_size+0xff/0x256 Aug 15 06:32:51 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:32:51 oleg133-client kernel: ? _raw_spin_unlock+0x12/0x30 Aug 15 06:32:51 oleg133-client kernel: do_sys_openat2+0x2b4/0x410 Aug 15 06:32:51 oleg133-client kernel: do_sys_open+0x73/0xa0 Aug 15 06:32:51 oleg133-client kernel: __x64_sys_openat+0x24/0x30 Aug 15 06:32:51 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:32:51 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:32:51 oleg133-client kernel: RIP: 0033:0x7fe02c20e332 Aug 15 06:32:51 oleg133-client kernel: Code: Unable to access opcode bytes at RIP 0x7fe02c20e308. Aug 15 06:32:51 oleg133-client kernel: RSP: 002b:00007fff45ac4b90 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 Aug 15 06:32:51 oleg133-client kernel: RAX: ffffffffffffffda RBX: 00000000000f0290 RCX: 00007fe02c20e332 Aug 15 06:32:51 oleg133-client kernel: RDX: 0000000000000042 RSI: 00007fff45ac4c60 RDI: 00000000ffffff9c Aug 15 06:32:51 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:32:51 oleg133-client kernel: R10: 00000000000001b6 R11: 0000000000000246 R12: 00007fff45ac4c60 Aug 15 06:32:51 oleg133-client kernel: R13: 00000000ffffffff R14: 000000000000015f R15: 0000000000000000 Aug 15 06:32:51 oleg133-client kernel: INFO: task writemany:36105 blocked for more than 120 seconds. Aug 15 06:32:51 oleg133-client kernel: Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:51 oleg133-client kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Aug 15 06:32:51 oleg133-client kernel: task:writemany state:D stack:0 pid:36105 ppid:36100 flags:0x80000004 Aug 15 06:32:51 oleg133-client kernel: Call Trace: Aug 15 06:32:51 oleg133-client kernel: __schedule+0x351/0xcb0 Aug 15 06:32:51 oleg133-client kernel: schedule+0xc0/0x180 Aug 15 06:32:51 oleg133-client kernel: schedule_preempt_disabled+0x21/0x40 Aug 15 06:32:51 oleg133-client kernel: rwsem_down_write_slowpath+0x5d7/0xa40 Aug 15 06:32:51 oleg133-client kernel: ? __might_sleep+0x59/0xc0 Aug 15 06:32:51 oleg133-client kernel: down_write+0x80/0xd0 Aug 15 06:32:51 oleg133-client kernel: do_unlinkat+0x184/0x460 Aug 15 06:32:51 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:32:51 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:32:51 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:32:51 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:32:51 oleg133-client kernel: Code: Unable to access opcode bytes at RIP 0x7fe02c1190e1. Aug 15 06:32:51 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:32:51 oleg133-client kernel: RAX: ffffffffffffffda RBX: 000000000058d508 RCX: 00007fe02c11910b Aug 15 06:32:51 oleg133-client kernel: RDX: 0000000000000038 RSI: 00007fff45ac5060 RDI: 00007fff45ac4c60 Aug 15 06:32:51 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1228 R09: 00007fe02c4b1260 Aug 15 06:32:51 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:32:51 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:32:51 oleg133-client kernel: INFO: task writemany:36106 blocked for more than 120 seconds. Aug 15 06:32:51 oleg133-client kernel: Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:51 oleg133-client kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Aug 15 06:32:51 oleg133-client kernel: task:writemany state:D stack:0 pid:36106 ppid:36100 flags:0x80000004 Aug 15 06:32:51 oleg133-client kernel: Call Trace: Aug 15 06:32:51 oleg133-client kernel: __schedule+0x351/0xcb0 Aug 15 06:32:51 oleg133-client kernel: schedule+0xc0/0x180 Aug 15 06:32:51 oleg133-client kernel: schedule_preempt_disabled+0x21/0x40 Aug 15 06:32:51 oleg133-client kernel: rwsem_down_write_slowpath+0x5d7/0xa40 Aug 15 06:32:51 oleg133-client kernel: ? __might_sleep+0x59/0xc0 Aug 15 06:32:51 oleg133-client kernel: down_write+0x80/0xd0 Aug 15 06:32:51 oleg133-client kernel: do_unlinkat+0x184/0x460 Aug 15 06:32:51 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:32:51 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:32:51 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:32:51 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:32:51 oleg133-client kernel: Code: Unable to access opcode bytes at RIP 0x7fe02c1190e1. Aug 15 06:32:51 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:32:51 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000c97218 RCX: 00007fe02c11910b Aug 15 06:32:51 oleg133-client kernel: RDX: 0000000000000038 RSI: 00007fff45ac5060 RDI: 00007fff45ac4c60 Aug 15 06:32:51 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b122c R09: 00007fe02c4b1260 Aug 15 06:32:51 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:32:51 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:32:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:32:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:32:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:32:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:56 oleg133-client kernel: Modules linked in: Aug 15 06:32:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:32:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:32:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:32:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 Aug 15 06:32:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:32:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:32:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:32:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:32:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:32:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:32:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:32:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:32:56 oleg133-client kernel: Call Trace: Aug 15 06:32:56 oleg133-client kernel: Aug 15 06:32:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:32:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:32:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:32:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:32:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:32:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:32:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:32:56 oleg133-client kernel: Aug 15 06:32:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:32:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:32:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:32:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:32:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:32:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:32:56 oleg133-client kernel: lustre(O) Aug 15 06:32:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:32:56 oleg133-client kernel: osp(O) Aug 15 06:32:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:32:56 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:32:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:32:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:32:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:32:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:32:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:32:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 Aug 15 06:32:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:32:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:32:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:32:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:32:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:32:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:32:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:32:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:32:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:32:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:32:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:32:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:32:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:32:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:32:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:32:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:32:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:32:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:32:57 oleg133-client kernel: ORIG_RAX: ffffffffffffff13 Aug 15 06:32:57 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:32:57 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:32:57 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:32:57 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:32:57 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:32:57 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:32:57 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:32:57 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:32:57 oleg133-client kernel: Call Trace: Aug 15 06:32:57 oleg133-client kernel: Aug 15 06:32:57 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:32:57 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:32:57 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:32:57 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:32:57 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:32:57 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:32:57 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:32:57 oleg133-client kernel: Aug 15 06:32:57 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:32:57 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:32:57 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:32:57 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:32:57 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:32:57 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:32:57 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:32:57 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:32:57 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:32:57 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:32:57 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:32:57 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:32:57 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:32:57 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:32:57 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:32:57 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:32:57 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:32:57 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:32:57 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:33:08 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:33:08 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:33:08 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:33:08 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:33:08 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:33:08 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:33:08 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:33:08 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:33:08 oleg133-client kernel: RAX: 0000000000007a16 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:33:08 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:33:08 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:33:08 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:33:08 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:33:08 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:33:08 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:33:08 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:33:08 oleg133-client kernel: Call Trace: Aug 15 06:33:08 oleg133-client kernel: Aug 15 06:33:08 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:33:08 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:33:08 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:33:08 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:33:08 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:33:08 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:33:08 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:33:08 oleg133-client kernel: Aug 15 06:33:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:33:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:33:08 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:33:08 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:33:08 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:33:08 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:33:08 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:33:08 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:33:08 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:33:08 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:33:08 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:33:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:33:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:33:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr Aug 15 06:33:24 oleg133-client kernel: Modules linked in: Aug 15 06:33:24 oleg133-client kernel: i2c_piix4 Aug 15 06:33:24 oleg133-client kernel: lustre(O) Aug 15 06:33:24 oleg133-client kernel: squashfs Aug 15 06:33:24 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:33:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:33:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:33:24 oleg133-client kernel: ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:33:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:33:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:33:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:33:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:33:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:33:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:33:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:33:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:33:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:33:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:33:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:33:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:33:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:33:24 oleg133-client kernel: Call Trace: Aug 15 06:33:24 oleg133-client kernel: Aug 15 06:33:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:33:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:33:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:33:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:33:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:33:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:33:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:33:24 oleg133-client kernel: Aug 15 06:33:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:33:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:33:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:33:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:33:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:33:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:33:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:33:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:33:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:33:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:33:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:33:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:33:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:33:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:33:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:33:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:33:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:33:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:33:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:33:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:33:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:33:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:33:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:33:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:33:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:33:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:33:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:33:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:33:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:33:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:33:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:33:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:33:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:33:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:33:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:33:24 oleg133-client kernel: Call Trace: Aug 15 06:33:24 oleg133-client kernel: Aug 15 06:33:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:33:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:33:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:33:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:33:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:33:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:33:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:33:24 oleg133-client kernel: Aug 15 06:33:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:33:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:33:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:33:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:33:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:33:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:33:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:33:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:33:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:33:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:33:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:33:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:33:36 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:33:36 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:33:36 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:33:36 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:33:36 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:33:36 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:33:36 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:33:36 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 Aug 15 06:33:36 oleg133-client kernel: RAX: 0000000000007a68 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:33:36 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:33:36 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:33:36 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:33:36 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:33:36 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:33:36 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:33:36 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:33:36 oleg133-client kernel: Call Trace: Aug 15 06:33:36 oleg133-client kernel: Aug 15 06:33:36 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:33:36 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:33:36 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:33:36 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:33:36 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:33:36 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:33:36 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:33:36 oleg133-client kernel: Aug 15 06:33:36 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:33:36 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:33:36 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:33:36 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:33:36 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:33:36 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:33:36 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:33:36 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:33:36 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:33:36 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:33:36 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:33:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:33:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:33:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:33:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:33:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:33:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:33:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:33:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:33:52 oleg133-client kernel: RAX: 0000000000005d08 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:33:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:33:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:33:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:33:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:33:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:33:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:33:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:33:52 oleg133-client kernel: Call Trace: Aug 15 06:33:52 oleg133-client kernel: Aug 15 06:33:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:33:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:33:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:33:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:33:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:33:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:33:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:33:52 oleg133-client kernel: Aug 15 06:33:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:33:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:33:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:33:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:33:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:33:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:33:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:33:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:33:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:33:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:33:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:33:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:33:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:33:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:33:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:33:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:33:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:33:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:33:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:33:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:33:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:33:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:33:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:33:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:33:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:33:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:33:53 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:33:53 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:33:53 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:33:53 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:33:53 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:33:53 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:33:53 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:33:53 oleg133-client kernel: RDX: 000000000000271e RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:33:53 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:33:53 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:33:53 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:33:53 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:33:53 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:33:53 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:33:53 oleg133-client kernel: Call Trace: Aug 15 06:33:53 oleg133-client kernel: Aug 15 06:33:53 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:33:53 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:33:53 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:33:53 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:33:53 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:33:53 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:33:53 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:33:53 oleg133-client kernel: Aug 15 06:33:53 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:33:53 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:33:53 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:33:53 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:33:53 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:33:53 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:33:53 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:33:53 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:33:53 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:33:53 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:33:53 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:33:53 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:33:53 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:33:53 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:33:53 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:33:53 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:33:53 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:33:53 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:33:54 oleg133-client systemd[1]: Starting dnf makecache... Aug 15 06:34:02 oleg133-client dnf[36980]: Failed determining last makecache time. Aug 15 06:34:02 oleg133-client dnf[36980]: [Errno 30] Read-only file system: '/var/cache/dnf/metadata_lock.pid' Aug 15 06:34:03 oleg133-client dnf[36980]: Failed to store expired repos cache: [Errno 30] Read-only file system: '/var/cache/dnf/expired_repos.json' Aug 15 06:34:03 oleg133-client systemd[1]: dnf-makecache.service: Main process exited, code=exited, status=1/FAILURE Aug 15 06:34:03 oleg133-client systemd[1]: dnf-makecache.service: Failed with result 'exit-code'. Aug 15 06:34:03 oleg133-client systemd[1]: Failed to start dnf makecache. Aug 15 06:34:04 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:34:04 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:34:04 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:34:04 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:34:04 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:34:04 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:34:04 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:34:04 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:34:04 oleg133-client kernel: RAX: 0000000000002a19 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:34:04 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:34:04 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:34:04 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:34:04 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:34:04 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:34:04 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:34:04 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:34:04 oleg133-client kernel: Call Trace: Aug 15 06:34:04 oleg133-client kernel: Aug 15 06:34:04 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:34:04 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:34:04 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:34:04 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:34:04 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:34:04 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:34:04 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:34:04 oleg133-client kernel: Aug 15 06:34:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:34:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:34:04 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:34:04 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:34:04 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:34:04 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:34:04 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:34:04 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:34:04 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:34:04 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:34:04 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:34:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:34:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:34:20 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:34:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:34:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:34:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:34:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:34:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:34:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:34:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:34:20 oleg133-client kernel: RDX: 000000000000793a RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:34:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:34:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:34:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:34:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:34:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:34:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:34:20 oleg133-client kernel: Call Trace: Aug 15 06:34:20 oleg133-client kernel: Aug 15 06:34:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:34:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:34:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:34:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:34:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:34:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:34:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:34:20 oleg133-client kernel: Aug 15 06:34:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:34:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:34:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:34:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:34:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:34:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:34:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:34:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:34:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:34:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:34:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:34:20 oleg133-client kernel: osp(O) Aug 15 06:34:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:34:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:34:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:34:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:34:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:34:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:34:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:34:21 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:34:21 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:34:21 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:34:21 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:34:21 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:34:21 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:34:21 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:34:21 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:34:21 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:34:21 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:34:21 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:34:21 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:34:21 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:34:21 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:34:21 oleg133-client kernel: Call Trace: Aug 15 06:34:21 oleg133-client kernel: Aug 15 06:34:21 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:34:21 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:34:21 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:34:21 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:34:21 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:34:21 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:34:21 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:34:21 oleg133-client kernel: Aug 15 06:34:21 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:34:21 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:34:21 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:34:21 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:34:21 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:34:21 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:34:21 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:34:21 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:34:21 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:34:21 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:34:21 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:34:21 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:34:21 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:34:21 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:34:21 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:34:21 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:34:21 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:34:21 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:34:21 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:34:21 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:34:21 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:34:21 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:34:21 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:34:21 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:34:21 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:34:21 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:34:21 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:34:32 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:34:32 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:34:32 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:34:32 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:34:32 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:34:32 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:34:32 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:34:32 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 Aug 15 06:34:32 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:34:32 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:34:32 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:34:32 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:34:32 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:34:32 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:34:32 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:34:32 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:34:32 oleg133-client kernel: Call Trace: Aug 15 06:34:32 oleg133-client kernel: Aug 15 06:34:32 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:34:32 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:34:32 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:34:32 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:34:32 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:34:32 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:34:32 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:34:32 oleg133-client kernel: Aug 15 06:34:32 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:34:32 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:34:32 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:34:32 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:34:32 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:34:32 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:34:32 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:34:32 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:34:32 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:34:32 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:34:32 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:34:32 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:34:32 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:34:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:34:34 oleg133-client kernel: rcu: #0111-....: (257989 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=76875 Aug 15 06:34:34 oleg133-client kernel: #011(t=420017 jiffies g=278549 q=3951) Aug 15 06:34:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:34:34 oleg133-client kernel: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 Aug 15 06:34:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:34:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:34:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:34:34 oleg133-client kernel: Call Trace: Aug 15 06:34:34 oleg133-client kernel: Aug 15 06:34:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:34:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:34:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:34:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:34:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:34:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:34:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:34:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:34:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:34:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:34:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:34:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:34:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:34:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:34:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:34:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:34:34 oleg133-client kernel: Aug 15 06:34:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:34:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:34:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:34:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:34:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:34:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:34:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:34:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:34:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:34:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:34:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:34:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:34:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:34:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:34:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:34:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:34:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:34:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:35:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:35:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:35:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:35:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:35:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:35:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:35:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:35:00 oleg133-client kernel: RAX: 0000000000000d26 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:35:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:35:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:35:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:35:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:35:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:35:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:35:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:35:00 oleg133-client kernel: Call Trace: Aug 15 06:35:00 oleg133-client kernel: Aug 15 06:35:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:35:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:35:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:35:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:35:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:35:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:35:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:35:00 oleg133-client kernel: Aug 15 06:35:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:35:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:35:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:35:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:35:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:35:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:35:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:35:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:35:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:35:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:35:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:35:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:35:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:35:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:35:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:35:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:35:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:35:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:35:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:35:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:35:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:35:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:35:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:35:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:35:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:35:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:35:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:35:00 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:35:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:35:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:35:00 oleg133-client kernel: RDX: 00000000000005e4 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:35:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:35:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:35:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:35:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:35:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:35:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:35:00 oleg133-client kernel: Call Trace: Aug 15 06:35:00 oleg133-client kernel: Aug 15 06:35:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:35:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:35:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:35:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:35:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:35:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:35:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:35:00 oleg133-client kernel: Aug 15 06:35:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:35:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:35:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:35:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:35:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:35:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:35:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:35:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:35:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:35:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:35:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:35:01 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:35:01 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:35:01 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:35:01 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:35:01 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:01 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:35:01 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:35:01 oleg133-client kernel: RAX: 0000000000001286 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:35:01 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:35:01 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:35:01 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:35:01 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:35:01 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:35:01 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:35:01 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:35:01 oleg133-client kernel: Call Trace: Aug 15 06:35:01 oleg133-client kernel: Aug 15 06:35:01 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:35:01 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:35:01 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:35:01 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:35:01 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:35:01 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:35:01 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:35:01 oleg133-client kernel: Aug 15 06:35:01 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:01 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:35:01 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:35:01 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:35:01 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:35:01 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:35:01 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:35:01 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:35:01 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:35:01 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:35:01 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:35:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:35:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:35:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:35:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:35:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:35:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:35:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:35:28 oleg133-client kernel: RAX: 0000000000007e26 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:35:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:35:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:35:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:35:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:35:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:35:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:35:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:35:28 oleg133-client kernel: Call Trace: Aug 15 06:35:28 oleg133-client kernel: Aug 15 06:35:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:35:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:35:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:35:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:35:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:35:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:35:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:35:28 oleg133-client kernel: Aug 15 06:35:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:35:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:35:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:35:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:35:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:35:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:35:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:35:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:35:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:35:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:35:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:35:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:35:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:35:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:35:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:35:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:35:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:35:28 oleg133-client kernel: Call Trace: Aug 15 06:35:28 oleg133-client kernel: Aug 15 06:35:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:35:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:35:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:35:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:35:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:35:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:35:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:35:28 oleg133-client kernel: Aug 15 06:35:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:35:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:35:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:35:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:35:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:35:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:35:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:35:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:35:28 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:35:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:35:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:35:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:35:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:35:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:35:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:35:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:35:28 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:35:28 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:35:28 oleg133-client kernel: RAX: 0000000000003d8e RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:35:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:35:28 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:35:28 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:35:28 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:35:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:35:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:35:28 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:35:28 oleg133-client kernel: Call Trace: Aug 15 06:35:28 oleg133-client kernel: Aug 15 06:35:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:35:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:35:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:35:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:35:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:35:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:35:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:35:28 oleg133-client kernel: Aug 15 06:35:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:35:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:35:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:35:28 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:35:28 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:35:28 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:35:28 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:35:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:35:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:35:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:35:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:35:29 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:35:29 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:35:29 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:35:29 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:35:29 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:35:29 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:35:29 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:35:29 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:35:29 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:35:29 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:35:29 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:35:29 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:35:29 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:35:29 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:35:29 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:35:29 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:35:29 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:35:29 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:35:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:35:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:35:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:35:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:35:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:35:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:35:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) Aug 15 06:35:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:56 oleg133-client kernel: ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) Aug 15 06:35:56 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:35:56 oleg133-client kernel: libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 Aug 15 06:35:56 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 Aug 15 06:35:56 oleg133-client kernel: dns_resolver intel_rapl_msr Aug 15 06:35:56 oleg133-client kernel: ORIG_RAX: ffffffffffffff13 Aug 15 06:35:56 oleg133-client kernel: intel_rapl_common Aug 15 06:35:56 oleg133-client kernel: RAX: 0000000000007b4c RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:35:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:35:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:35:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:35:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:35:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:35:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:35:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:35:56 oleg133-client kernel: Call Trace: Aug 15 06:35:56 oleg133-client kernel: Aug 15 06:35:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:35:56 oleg133-client kernel: sb_edac rapl Aug 15 06:35:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:35:56 oleg133-client kernel: pcspkr i2c_piix4 squashfs ata_generic Aug 15 06:35:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:35:56 oleg133-client kernel: crct10dif_pclmul crc32_pclmul crc32c_intel Aug 15 06:35:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:35:56 oleg133-client kernel: ghash_clmulni_intel ata_piix serio_raw Aug 15 06:35:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:35:56 oleg133-client kernel: libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:35:56 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:35:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:35:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:35:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:35:56 oleg133-client kernel: Aug 15 06:35:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:35:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:35:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:35:56 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:35:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:56 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:35:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:35:56 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:35:56 oleg133-client kernel: RAX: 0000000000007f21 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:35:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:35:56 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:35:56 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:35:56 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:35:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:35:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:35:56 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:35:56 oleg133-client kernel: Call Trace: Aug 15 06:35:56 oleg133-client kernel: Aug 15 06:35:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:35:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:35:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:35:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:35:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:35:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:35:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:35:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:35:56 oleg133-client kernel: Aug 15 06:35:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:35:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:35:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:35:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:35:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:35:56 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:35:56 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:35:56 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:35:56 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:35:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:35:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:35:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:35:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:35:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:35:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:35:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:35:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:35:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:35:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:35:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:35:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:35:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:35:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:35:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:35:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:35:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:35:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:35:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:35:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:35:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:35:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:35:57 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:35:57 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:35:57 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:35:57 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:35:57 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:35:57 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:35:57 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:35:57 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:35:57 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:35:57 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:35:57 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:35:57 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:35:57 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:35:57 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:35:57 oleg133-client kernel: Call Trace: Aug 15 06:35:57 oleg133-client kernel: Aug 15 06:35:57 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:35:57 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:35:57 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:35:57 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:35:57 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:35:57 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:35:57 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:35:57 oleg133-client kernel: Aug 15 06:35:57 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:35:57 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:35:57 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:35:57 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:35:57 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:35:57 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:35:57 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:35:57 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:35:57 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:35:57 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:35:57 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:35:57 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:35:57 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:35:57 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:35:57 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:35:57 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:35:57 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:35:57 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:35:57 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:36:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:36:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:36:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) Aug 15 06:36:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:36:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:36:24 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:36:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:36:24 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:36:24 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:36:24 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:36:24 oleg133-client kernel: RAX: 0000000000005e8a RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:36:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:36:24 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:36:24 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:36:24 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:36:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:36:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:36:24 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:36:24 oleg133-client kernel: Call Trace: Aug 15 06:36:24 oleg133-client kernel: Aug 15 06:36:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:36:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:36:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:36:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:36:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:36:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:36:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:36:24 oleg133-client kernel: Aug 15 06:36:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:36:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:36:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:36:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:36:24 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:36:24 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:36:24 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:36:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:36:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:36:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:36:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:36:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:36:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:36:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:36:24 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:36:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:36:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:36:24 oleg133-client kernel: RDX: 000000000000566f RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:36:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:36:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:36:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:36:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:36:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:36:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:36:24 oleg133-client kernel: Call Trace: Aug 15 06:36:24 oleg133-client kernel: Aug 15 06:36:24 oleg133-client kernel: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:36:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:36:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:36:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:36:24 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:36:24 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:36:24 oleg133-client kernel: RAX: 0000000000004ac7 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:36:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:36:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:36:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:36:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:36:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:36:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:36:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:36:24 oleg133-client kernel: Call Trace: Aug 15 06:36:24 oleg133-client kernel: Aug 15 06:36:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:36:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:36:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:36:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:36:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:36:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:36:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:36:24 oleg133-client kernel: Aug 15 06:36:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:36:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:36:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:36:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:36:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:36:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:36:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:36:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:36:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:36:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:36:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:36:24 oleg133-client kernel: Aug 15 06:36:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:36:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:36:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:36:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:36:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:36:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:36:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:36:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:36:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:36:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:36:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:36:25 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:36:25 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:36:25 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:36:25 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:36:25 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:36:25 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:36:25 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:36:25 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:36:25 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:36:25 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:36:25 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:36:25 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:36:25 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:36:25 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:36:25 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:36:25 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:36:25 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:36:25 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:36:25 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:36:25 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:36:25 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:36:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:36:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:36:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:36:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:36:52 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:36:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:36:52 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:36:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:36:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:36:52 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:36:52 oleg133-client kernel: RAX: 0000000000006b4a RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:36:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:36:52 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:36:52 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:36:52 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:36:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:36:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:36:52 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:36:52 oleg133-client kernel: Call Trace: Aug 15 06:36:52 oleg133-client kernel: Aug 15 06:36:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:36:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:36:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:36:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:36:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:36:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:36:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:36:52 oleg133-client kernel: Aug 15 06:36:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:36:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:36:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:36:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:36:52 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:36:52 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:36:52 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:36:52 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:36:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:36:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:36:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:36:52 oleg133-client kernel: Modules linked in: Aug 15 06:36:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:36:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:36:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:36:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:36:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:36:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:36:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:36:52 oleg133-client kernel: RDX: 000000000000179a RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:36:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:36:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:36:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:36:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:36:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:36:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:36:52 oleg133-client kernel: Call Trace: Aug 15 06:36:52 oleg133-client kernel: Aug 15 06:36:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:36:52 oleg133-client kernel: lustre(O) Aug 15 06:36:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:36:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:36:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:36:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:36:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:36:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:36:52 oleg133-client kernel: Aug 15 06:36:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:36:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:36:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:36:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:36:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:36:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:36:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:36:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:36:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:36:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:36:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:36:52 oleg133-client kernel: osp(O) Aug 15 06:36:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:36:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:36:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:36:52 oleg133-client kernel: ofd(O) Aug 15 06:36:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:36:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:36:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:36:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:36:53 oleg133-client kernel: lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:36:53 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:36:53 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:36:53 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:36:53 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:36:53 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:36:53 oleg133-client kernel: RAX: 0000000000000b51 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:36:53 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:36:53 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:36:53 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:36:53 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:36:53 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:36:53 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:36:53 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:36:53 oleg133-client kernel: Call Trace: Aug 15 06:36:53 oleg133-client kernel: Aug 15 06:36:53 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:36:53 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:36:53 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:36:53 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:36:53 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:36:53 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:36:53 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:36:53 oleg133-client kernel: Aug 15 06:36:53 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:36:53 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:36:53 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:36:53 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:36:53 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:36:53 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:36:53 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:36:53 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:36:53 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:36:53 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:36:53 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:36:53 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:36:53 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:36:53 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:36:53 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:36:53 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:36:53 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:36:53 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:36:53 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:36:53 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:36:53 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:36:53 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:36:53 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:36:53 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:36:53 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:36:53 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:37:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:37:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:37:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:37:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:37:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:37:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:37:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:37:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:37:20 oleg133-client kernel: RAX: 0000000000005910 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:37:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:37:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:37:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:37:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:37:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:37:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:37:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:37:20 oleg133-client kernel: Call Trace: Aug 15 06:37:20 oleg133-client kernel: Aug 15 06:37:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:37:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:37:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:37:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:37:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:37:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:37:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:37:20 oleg133-client kernel: Aug 15 06:37:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:37:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:37:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:37:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:37:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:37:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:37:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:37:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:37:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:37:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:37:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:37:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:37:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:37:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:37:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:37:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:37:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:37:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:37:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:37:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:37:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:37:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:37:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:37:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:37:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:37:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:37:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:37:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:37:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:37:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:37:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:37:20 oleg133-client kernel: RDX: 00000000000007e9 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:37:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:37:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:37:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:37:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:37:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:37:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:37:20 oleg133-client kernel: Call Trace: Aug 15 06:37:20 oleg133-client kernel: Aug 15 06:37:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:37:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:37:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:37:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:37:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:37:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:37:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:37:20 oleg133-client kernel: Aug 15 06:37:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:37:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:37:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:37:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:37:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:37:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:37:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:37:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:37:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:37:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:37:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:37:21 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:37:21 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:37:21 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:37:21 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:37:21 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:37:21 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:37:21 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 Aug 15 06:37:21 oleg133-client kernel: RAX: 0000000000004189 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:37:21 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:37:21 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:37:21 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:37:21 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:37:21 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:37:21 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:37:21 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:37:21 oleg133-client kernel: Call Trace: Aug 15 06:37:21 oleg133-client kernel: Aug 15 06:37:21 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:37:21 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:37:21 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:37:21 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:37:21 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:37:21 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:37:21 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:37:21 oleg133-client kernel: Aug 15 06:37:21 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:37:21 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:37:21 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:37:21 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:37:21 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:37:21 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:37:21 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:37:21 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:37:21 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:37:21 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:37:21 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:37:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:37:34 oleg133-client kernel: rcu: #0111-....: (385393 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=112704 Aug 15 06:37:34 oleg133-client kernel: #011(t=600020 jiffies g=278549 q=4605) Aug 15 06:37:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:37:34 oleg133-client kernel: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 Aug 15 06:37:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:37:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:37:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:37:34 oleg133-client kernel: Call Trace: Aug 15 06:37:34 oleg133-client kernel: Aug 15 06:37:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:37:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:37:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:37:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:37:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:37:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:37:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:37:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:37:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:37:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:37:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:37:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:37:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:37:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:37:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:37:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:37:34 oleg133-client kernel: Aug 15 06:37:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:37:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:37:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:37:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:37:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:37:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:37:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:37:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:37:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:37:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:37:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:37:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:37:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:37:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:37:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:37:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:37:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:37:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:37:48 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:37:48 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:37:48 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:37:48 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:37:48 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:37:48 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:37:48 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:37:48 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:37:48 oleg133-client kernel: RAX: 0000000000003abe RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:37:48 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:37:48 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:37:48 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:37:48 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:37:48 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:37:48 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:37:48 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:37:48 oleg133-client kernel: Call Trace: Aug 15 06:37:48 oleg133-client kernel: Aug 15 06:37:48 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:37:48 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:37:48 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:37:48 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:37:48 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:37:48 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:37:48 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:37:48 oleg133-client kernel: Aug 15 06:37:48 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:37:48 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:37:48 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:37:48 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:37:48 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:37:48 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:37:48 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:37:48 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:37:48 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:37:48 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:37:48 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:38:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:38:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:38:00 oleg133-client kernel: Modules linked in: Aug 15 06:38:00 oleg133-client kernel: Modules linked in: Aug 15 06:38:00 oleg133-client kernel: lustre(O) osp(O) Aug 15 06:38:00 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:38:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:38:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:38:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:38:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:38:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:38:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:38:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:38:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:38:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:38:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:38:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:38:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:38:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:38:00 oleg133-client kernel: Call Trace: Aug 15 06:38:00 oleg133-client kernel: Aug 15 06:38:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:38:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:38:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:38:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:38:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:38:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:38:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:38:00 oleg133-client kernel: Aug 15 06:38:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:38:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:38:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:38:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:38:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:38:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:38:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:38:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:38:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:38:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:38:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:38:00 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:38:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:38:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:38:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:38:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:38:00 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:38:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:38:00 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:38:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:38:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:38:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:38:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:38:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:38:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:38:00 oleg133-client kernel: Call Trace: Aug 15 06:38:00 oleg133-client kernel: Aug 15 06:38:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:38:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:38:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:38:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:38:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:38:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:38:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:38:00 oleg133-client kernel: Aug 15 06:38:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:38:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:38:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:38:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:38:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:38:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:38:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:38:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:38:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:38:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:38:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:38:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:38:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:38:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:38:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:38:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:38:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:38:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:38:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:38:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:38:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:38:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:38:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:38:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:38:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:38:16 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:38:16 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:38:16 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:38:16 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:38:16 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:38:16 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:38:16 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:38:16 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:38:16 oleg133-client kernel: RAX: 0000000000004652 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:38:16 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:38:16 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:38:16 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:38:16 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:38:16 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:38:16 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:38:16 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:38:16 oleg133-client kernel: Call Trace: Aug 15 06:38:16 oleg133-client kernel: Aug 15 06:38:16 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:38:16 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:38:16 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:38:16 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:38:16 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:38:16 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:38:16 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:38:16 oleg133-client kernel: Aug 15 06:38:16 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:38:16 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:38:16 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:38:16 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:38:16 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:38:16 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:38:16 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:38:16 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:38:16 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:38:16 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:38:16 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:38:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:38:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:38:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:38:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:38:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:38:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:38:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:38:28 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:38:28 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:38:28 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:38:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:38:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:38:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:38:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:38:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:38:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:38:28 oleg133-client kernel: Call Trace: Aug 15 06:38:28 oleg133-client kernel: Aug 15 06:38:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:38:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:38:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:38:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:38:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:38:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:38:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:38:28 oleg133-client kernel: Aug 15 06:38:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:38:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:38:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:38:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:38:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:38:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:38:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:38:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:38:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:38:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:38:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:38:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:38:28 oleg133-client kernel: RDX: 000000000000321c RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:38:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:38:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:38:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:38:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:38:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:38:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:38:28 oleg133-client kernel: Call Trace: Aug 15 06:38:28 oleg133-client kernel: Aug 15 06:38:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:38:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:38:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:38:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:38:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:38:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:38:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:38:28 oleg133-client kernel: Aug 15 06:38:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:38:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:38:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:38:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:38:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:38:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:38:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:38:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:38:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:38:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:38:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:38:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:38:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:38:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:38:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:38:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:38:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:38:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:38:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:38:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:38:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:38:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:38:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:38:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:38:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:38:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:38:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:38:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:38:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:38:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:38:44 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:38:44 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:38:44 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:38:44 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:38:44 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:38:44 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:38:44 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:38:44 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:38:44 oleg133-client kernel: RAX: 000000000000758d RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:38:44 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:38:44 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:38:44 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:38:44 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:38:44 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:38:44 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:38:44 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:38:44 oleg133-client kernel: Call Trace: Aug 15 06:38:44 oleg133-client kernel: Aug 15 06:38:44 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:38:44 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:38:44 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:38:44 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:38:44 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:38:44 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:38:44 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:38:44 oleg133-client kernel: Aug 15 06:38:44 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:38:44 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:38:44 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:38:44 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:38:44 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:38:44 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:38:44 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:38:44 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:38:44 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:38:44 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:38:44 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:38:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:38:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:38:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:38:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:38:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:38:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:38:56 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:38:56 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:38:56 oleg133-client kernel: RAX: 0000000000007604 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:38:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:38:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:38:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:38:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:38:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:38:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:38:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:38:56 oleg133-client kernel: Call Trace: Aug 15 06:38:56 oleg133-client kernel: Aug 15 06:38:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:38:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:38:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:38:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:38:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:38:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:38:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:38:56 oleg133-client kernel: Aug 15 06:38:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:38:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:38:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:38:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:38:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:38:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:38:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:38:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:38:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:38:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:38:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:38:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:38:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:38:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:38:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:38:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:38:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:38:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:38:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:38:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:38:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:38:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:38:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:38:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:38:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:38:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:38:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:38:56 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:38:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:38:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:38:56 oleg133-client kernel: RDX: 000000000000670b RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:38:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:38:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:38:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:38:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:38:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:38:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:38:56 oleg133-client kernel: Call Trace: Aug 15 06:38:56 oleg133-client kernel: Aug 15 06:38:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:38:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:38:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:38:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:38:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:38:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:38:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:38:56 oleg133-client kernel: Aug 15 06:38:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:38:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:38:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:38:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:38:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:38:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:38:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:38:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:38:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:38:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:38:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:39:12 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:39:12 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:39:12 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:39:12 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:39:12 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:39:12 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:39:12 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:39:12 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:39:12 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:39:12 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:39:12 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:39:12 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:39:12 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:39:12 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:39:12 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:39:12 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:39:12 oleg133-client kernel: Call Trace: Aug 15 06:39:12 oleg133-client kernel: Aug 15 06:39:12 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:39:12 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:39:12 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:39:12 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:39:12 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:39:12 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:39:12 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:39:12 oleg133-client kernel: Aug 15 06:39:12 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:39:12 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:39:12 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:39:12 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:39:12 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:39:12 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:39:12 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:39:12 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:39:12 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:39:12 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:39:12 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:39:12 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:39:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:39:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:39:24 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:39:24 oleg133-client kernel: Modules linked in: Aug 15 06:39:24 oleg133-client kernel: osp(O) ofd(O) lod(O) Aug 15 06:39:24 oleg133-client kernel: lustre(O) Aug 15 06:39:24 oleg133-client kernel: mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common Aug 15 06:39:24 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr Aug 15 06:39:24 oleg133-client kernel: sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:39:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:39:24 oleg133-client kernel: i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:39:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:39:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:39:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:39:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:39:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:39:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:39:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:39:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:39:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:39:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:39:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:39:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:39:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:39:24 oleg133-client kernel: Call Trace: Aug 15 06:39:24 oleg133-client kernel: Aug 15 06:39:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:39:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:39:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:39:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:39:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:39:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:39:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:39:24 oleg133-client kernel: Aug 15 06:39:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:39:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:39:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:39:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:39:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:39:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:39:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:39:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:39:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:39:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:39:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:39:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:39:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:39:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:39:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:39:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:39:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:39:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:39:24 oleg133-client kernel: Call Trace: Aug 15 06:39:24 oleg133-client kernel: Aug 15 06:39:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:39:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:39:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:39:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:39:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:39:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:39:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:39:24 oleg133-client kernel: Aug 15 06:39:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:39:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:39:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:39:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:39:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:39:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:39:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:39:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:39:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:39:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:39:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:39:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:39:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:39:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:39:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:39:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:39:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:39:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:39:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:39:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:39:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:39:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:39:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:39:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:39:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:39:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:39:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:39:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:39:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:39:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:39:40 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:39:40 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:39:40 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:39:40 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:39:40 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:39:40 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:39:40 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:39:40 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 Aug 15 06:39:40 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:39:40 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:39:40 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:39:40 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:39:40 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:39:40 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:39:40 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:39:40 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:39:40 oleg133-client kernel: Call Trace: Aug 15 06:39:40 oleg133-client kernel: Aug 15 06:39:40 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:39:40 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:39:40 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:39:40 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:39:40 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:39:40 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:39:40 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:39:40 oleg133-client kernel: Aug 15 06:39:40 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:39:40 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:39:40 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:39:40 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:39:40 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:39:40 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:39:40 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:39:40 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:39:40 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:39:40 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:39:40 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:39:40 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:39:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:39:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:39:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:39:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:39:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:39:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:39:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:39:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:39:52 oleg133-client kernel: RAX: 0000000000002b23 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:39:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:39:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:39:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:39:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:39:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:39:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:39:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:39:52 oleg133-client kernel: Call Trace: Aug 15 06:39:52 oleg133-client kernel: Aug 15 06:39:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:39:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:39:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:39:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:39:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:39:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:39:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:39:52 oleg133-client kernel: Aug 15 06:39:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:39:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:39:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:39:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:39:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:39:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:39:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:39:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:39:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:39:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:39:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:39:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:39:52 oleg133-client kernel: RDX: 0000000000007402 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:39:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:39:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:39:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:39:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:39:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:39:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:39:52 oleg133-client kernel: Call Trace: Aug 15 06:39:52 oleg133-client kernel: Aug 15 06:39:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:39:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:39:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:39:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:39:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:39:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:39:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:39:52 oleg133-client kernel: Aug 15 06:39:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:39:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:39:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:39:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:39:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:39:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:39:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:39:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:39:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:39:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:39:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:39:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:39:52 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:39:53 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:39:53 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:39:53 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:39:53 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:39:53 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:39:53 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:39:53 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:39:53 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:39:53 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:39:53 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:39:53 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:39:53 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:39:53 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:39:53 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:39:53 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:39:53 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:39:53 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:40:08 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:40:08 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:40:08 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:40:08 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:40:08 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:40:08 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:40:08 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:40:08 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:40:08 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:40:08 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:40:08 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:40:08 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:40:08 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:40:08 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:40:08 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:40:08 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:40:08 oleg133-client kernel: Call Trace: Aug 15 06:40:08 oleg133-client kernel: Aug 15 06:40:08 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:40:08 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:40:08 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:40:08 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:40:08 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:40:08 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:40:08 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:40:08 oleg133-client kernel: Aug 15 06:40:08 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:40:08 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:40:08 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:40:08 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:40:08 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:40:08 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:40:08 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:40:08 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:40:08 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:40:08 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:40:08 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:40:08 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:40:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:40:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:40:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) Aug 15 06:40:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:40:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:40:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:40:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:40:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:40:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:40:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:40:20 oleg133-client kernel: RDX: 0000000000005131 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:40:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:40:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:40:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:40:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:40:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:40:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:40:20 oleg133-client kernel: Call Trace: Aug 15 06:40:20 oleg133-client kernel: Aug 15 06:40:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:40:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:40:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:40:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:40:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:40:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:40:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:40:20 oleg133-client kernel: Aug 15 06:40:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:40:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:40:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:40:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:40:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:40:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:40:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:40:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:40:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:40:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:40:20 oleg133-client kernel: lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:40:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:40:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:40:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:40:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:40:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:40:20 oleg133-client kernel: RAX: 000000000000792f RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:40:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:40:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:40:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:40:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:40:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:40:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:40:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:40:20 oleg133-client kernel: Call Trace: Aug 15 06:40:20 oleg133-client kernel: Aug 15 06:40:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:40:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:40:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:40:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:40:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:40:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:40:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:40:20 oleg133-client kernel: Aug 15 06:40:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:40:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:40:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:40:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:40:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:40:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:40:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:40:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:40:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:40:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:40:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:40:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:40:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:40:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:40:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:40:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:40:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:40:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:40:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:40:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:40:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:40:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:40:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:40:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:40:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:40:34 oleg133-client kernel: rcu: #0111-....: (514040 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=148573 Aug 15 06:40:34 oleg133-client kernel: #011(t=780023 jiffies g=278549 q=5382) Aug 15 06:40:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:40:34 oleg133-client kernel: INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 3.136 msecs Aug 15 06:40:34 oleg133-client kernel: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 Aug 15 06:40:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:40:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:40:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:40:34 oleg133-client kernel: Call Trace: Aug 15 06:40:34 oleg133-client kernel: Aug 15 06:40:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:40:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:40:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:40:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:40:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:40:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:40:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:40:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:40:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:40:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:40:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:40:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:40:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:40:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:40:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:40:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:40:34 oleg133-client kernel: Aug 15 06:40:34 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:40:34 oleg133-client kernel: ? kvm_wait+0xcd/0xe0 Aug 15 06:40:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:40:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:40:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:40:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:40:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:40:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:40:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:40:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:40:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:40:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:40:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:40:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:40:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:40:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:40:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:40:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:40:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:40:36 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:40:36 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:40:36 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:40:36 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:40:36 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:40:36 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:40:36 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:40:36 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:40:36 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:40:36 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:40:36 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:40:36 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:40:36 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:40:36 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:40:36 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:40:36 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:40:36 oleg133-client kernel: Call Trace: Aug 15 06:40:36 oleg133-client kernel: Aug 15 06:40:36 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:40:36 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:40:36 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:40:36 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:40:36 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:40:36 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:40:36 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:40:36 oleg133-client kernel: Aug 15 06:40:36 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:40:36 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:40:36 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:40:36 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:40:36 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:40:36 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:40:36 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:40:36 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:40:36 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:40:36 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:40:36 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:40:36 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:41:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:41:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:41:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:41:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:41:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:41:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:41:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:41:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:41:00 oleg133-client kernel: RAX: 0000000000007f90 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:41:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:41:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:41:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:41:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:41:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:41:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:41:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:41:00 oleg133-client kernel: Call Trace: Aug 15 06:41:00 oleg133-client kernel: Aug 15 06:41:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:41:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:41:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:41:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:41:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:41:00 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:41:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:41:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:41:00 oleg133-client kernel: RDX: 0000000000007470 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:41:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:41:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:41:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:41:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:41:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:41:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:41:00 oleg133-client kernel: Call Trace: Aug 15 06:41:00 oleg133-client kernel: Aug 15 06:41:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:41:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:41:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:41:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:41:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:41:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:41:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:41:00 oleg133-client kernel: Aug 15 06:41:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:41:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:41:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:41:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:41:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:41:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:41:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:41:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:41:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:41:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:41:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:41:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:41:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:41:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:41:00 oleg133-client kernel: Aug 15 06:41:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:41:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:41:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:41:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:41:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:41:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:41:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:41:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:41:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:41:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:41:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:41:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:41:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:41:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:41:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:41:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:41:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:41:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:41:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:41:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:41:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:41:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:41:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:41:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:41:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:41:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:41:01 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:41:01 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:41:01 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:41:01 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:41:01 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:41:01 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:41:01 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:41:01 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:41:01 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:41:01 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:41:04 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:41:04 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:41:04 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:41:04 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:41:04 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:41:04 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:41:04 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:41:04 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:41:04 oleg133-client kernel: RAX: 0000000000007f84 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:41:04 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:41:04 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:41:04 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:41:04 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:41:04 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:41:04 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:41:04 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:41:04 oleg133-client kernel: Call Trace: Aug 15 06:41:04 oleg133-client kernel: Aug 15 06:41:04 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:41:04 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:41:04 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:41:04 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:41:04 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:41:04 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:41:04 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:41:04 oleg133-client kernel: Aug 15 06:41:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:41:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:41:04 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:41:04 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:41:04 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:41:04 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:41:04 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:41:04 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:41:04 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:41:04 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:41:04 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:41:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:41:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:41:28 oleg133-client kernel: Modules linked in: Aug 15 06:41:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:41:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:41:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:41:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:41:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:41:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:41:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:41:28 oleg133-client kernel: RDX: 0000000000004943 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:41:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:41:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:41:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:41:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:41:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:41:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:41:28 oleg133-client kernel: Call Trace: Aug 15 06:41:28 oleg133-client kernel: Aug 15 06:41:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:41:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:41:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:41:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:41:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:41:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:41:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:41:28 oleg133-client kernel: Aug 15 06:41:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:41:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:41:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:41:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:41:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: lustre(O) Aug 15 06:41:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:41:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:41:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:41:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:41:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:41:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:41:28 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:41:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:41:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:41:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:41:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:41:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:41:28 oleg133-client kernel: RAX: 0000000000007a89 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:41:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:41:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:41:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:41:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:41:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:41:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:41:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:41:28 oleg133-client kernel: Call Trace: Aug 15 06:41:28 oleg133-client kernel: Aug 15 06:41:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:41:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:41:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:41:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:41:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:41:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:41:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:41:28 oleg133-client kernel: Aug 15 06:41:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:41:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:41:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:41:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:41:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:41:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:41:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:41:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:41:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:41:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:41:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:41:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:41:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:41:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:41:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:41:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:41:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:41:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:41:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:41:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:41:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:41:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:41:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:41:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:41:32 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:41:32 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:41:32 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:41:32 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:41:32 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:41:32 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:41:32 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:41:32 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:41:32 oleg133-client kernel: RAX: 0000000000005967 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:41:32 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:41:32 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:41:32 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:41:32 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:41:32 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:41:32 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:41:32 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:41:32 oleg133-client kernel: Call Trace: Aug 15 06:41:32 oleg133-client kernel: Aug 15 06:41:32 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:41:32 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:41:32 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:41:32 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:41:32 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:41:32 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:41:32 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:41:32 oleg133-client kernel: Aug 15 06:41:32 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:41:32 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:41:32 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:41:32 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:41:32 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:41:32 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:41:32 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:41:32 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:41:32 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:41:32 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:41:32 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:41:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:41:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:41:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:41:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:41:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:41:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:41:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:41:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:41:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:41:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:41:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:41:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:41:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:41:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:41:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:41:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:41:56 oleg133-client kernel: Call Trace: Aug 15 06:41:56 oleg133-client kernel: Aug 15 06:41:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:41:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:41:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:41:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:41:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:41:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:41:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:41:56 oleg133-client kernel: Aug 15 06:41:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:41:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:41:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:41:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:41:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:41:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:41:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:41:56 oleg133-client kernel: osp(O) Aug 15 06:41:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: ofd(O) Aug 15 06:41:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:41:56 oleg133-client kernel: lod(O) Aug 15 06:41:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: mdt(O) Aug 15 06:41:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:41:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:41:56 oleg133-client kernel: mdd(O) Aug 15 06:41:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:41:56 oleg133-client kernel: mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:41:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:41:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:41:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:41:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:41:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:41:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:41:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:41:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:41:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:41:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:41:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:41:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:41:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:41:56 oleg133-client kernel: Call Trace: Aug 15 06:41:56 oleg133-client kernel: Aug 15 06:41:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:41:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:41:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:41:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:41:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:41:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:41:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:41:56 oleg133-client kernel: Aug 15 06:41:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:41:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:41:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:41:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:41:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:41:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:41:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:41:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:41:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:41:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:41:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:41:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:41:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:41:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:41:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:41:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:41:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:41:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:41:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:41:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:41:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:41:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:41:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:41:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:41:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:41:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:42:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:42:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:42:00 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:42:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:42:00 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:42:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:42:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:42:00 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 Aug 15 06:42:00 oleg133-client kernel: RAX: 0000000000004783 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:42:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:42:00 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:42:00 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:42:00 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:42:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:42:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:42:00 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:42:00 oleg133-client kernel: Call Trace: Aug 15 06:42:00 oleg133-client kernel: Aug 15 06:42:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:42:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:42:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:42:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:42:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:42:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:42:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:42:00 oleg133-client kernel: Aug 15 06:42:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:42:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:42:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:42:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:42:00 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:42:00 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:42:00 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:42:00 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:42:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:42:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:42:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:42:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:42:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:42:24 oleg133-client kernel: Modules linked in: Aug 15 06:42:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:42:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:42:24 oleg133-client kernel: lustre(O) Aug 15 06:42:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:42:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:42:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:42:24 oleg133-client kernel: osp(O) Aug 15 06:42:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:42:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:42:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:42:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:42:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:42:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:42:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:42:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:42:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:42:24 oleg133-client kernel: Call Trace: Aug 15 06:42:24 oleg133-client kernel: Aug 15 06:42:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:42:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:42:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:42:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:42:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:42:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:42:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:42:24 oleg133-client kernel: Aug 15 06:42:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:42:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:42:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:42:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:42:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:42:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:42:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:42:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:42:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:42:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:42:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:42:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:42:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:42:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:42:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:42:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:42:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:42:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:42:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:42:25 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:42:25 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:42:25 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:42:25 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:42:25 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:42:25 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 Aug 15 06:42:25 oleg133-client kernel: RAX: 0000000000001f0f RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:42:25 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:42:25 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:42:25 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:42:25 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:42:25 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:42:25 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:42:25 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:42:25 oleg133-client kernel: Call Trace: Aug 15 06:42:25 oleg133-client kernel: Aug 15 06:42:25 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:42:25 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:42:25 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:42:25 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:42:25 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:42:25 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:42:25 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:42:25 oleg133-client kernel: Aug 15 06:42:25 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:42:25 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:42:25 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:42:25 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:42:25 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:42:25 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:42:25 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:42:25 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:42:25 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:42:25 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:42:25 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:42:25 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:42:25 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:42:25 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:42:25 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:42:25 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:42:25 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:42:25 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:42:25 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:42:25 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:42:25 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:42:25 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:42:25 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:42:25 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:42:25 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:42:25 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:42:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:42:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:42:28 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:42:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:42:28 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:42:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:42:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:42:28 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:42:28 oleg133-client kernel: RAX: 0000000000007c90 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:42:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:42:28 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:42:28 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:42:28 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:42:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:42:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:42:28 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:42:28 oleg133-client kernel: Call Trace: Aug 15 06:42:28 oleg133-client kernel: Aug 15 06:42:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:42:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:42:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:42:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:42:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:42:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:42:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:42:28 oleg133-client kernel: Aug 15 06:42:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:42:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:42:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:42:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:42:28 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:42:28 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:42:28 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:42:28 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:42:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:42:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:42:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:42:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:42:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:42:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) Aug 15 06:42:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:42:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:42:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:42:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:42:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:42:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:42:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:42:52 oleg133-client kernel: RDX: 00000000000014e9 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:42:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:42:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:42:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:42:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:42:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:42:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:42:52 oleg133-client kernel: Call Trace: Aug 15 06:42:52 oleg133-client kernel: Aug 15 06:42:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:42:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:42:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:42:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:42:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:42:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:42:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:42:52 oleg133-client kernel: Aug 15 06:42:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:42:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:42:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:42:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:42:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:42:52 oleg133-client kernel: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:42:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:42:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:42:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:42:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:42:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:42:52 oleg133-client kernel: RAX: 000000000000770c RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:42:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:42:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:42:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:42:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:42:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:42:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:42:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:42:52 oleg133-client kernel: Call Trace: Aug 15 06:42:52 oleg133-client kernel: Aug 15 06:42:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:42:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:42:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:42:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:42:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:42:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:42:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:42:52 oleg133-client kernel: Aug 15 06:42:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:42:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:42:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:42:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:42:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:42:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:42:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:42:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:42:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:42:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:42:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:42:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:42:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:42:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:42:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:42:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:42:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:42:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:42:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:42:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:42:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:42:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:42:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:42:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:42:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:42:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:42:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:42:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:42:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:42:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:42:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:42:56 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:42:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:42:56 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:42:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:42:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:42:56 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:42:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:42:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:42:56 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:42:56 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:42:56 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:42:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:42:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:42:56 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:42:56 oleg133-client kernel: Call Trace: Aug 15 06:42:56 oleg133-client kernel: Aug 15 06:42:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:42:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:42:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:42:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:42:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:42:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:42:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:42:56 oleg133-client kernel: Aug 15 06:42:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:42:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:42:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:42:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:42:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:42:56 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:42:56 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:42:56 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:42:56 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:42:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:42:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:42:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:43:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:43:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:43:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:43:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:43:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:43:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:43:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:43:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:43:20 oleg133-client kernel: RAX: 0000000000003786 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:43:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:43:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:43:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:43:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:43:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:43:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:43:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:43:20 oleg133-client kernel: Call Trace: Aug 15 06:43:20 oleg133-client kernel: Aug 15 06:43:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:43:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:43:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:43:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:43:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:43:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:43:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:43:20 oleg133-client kernel: Aug 15 06:43:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:43:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:43:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:43:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:43:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:43:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:43:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:43:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:43:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:43:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) Aug 15 06:43:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:43:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:43:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:43:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:43:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:43:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:43:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:43:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:43:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:43:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:43:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:43:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:43:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:43:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:43:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:43:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:43:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:43:21 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:43:21 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:43:21 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:43:21 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:43:21 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:43:21 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:43:21 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:43:21 oleg133-client kernel: RDX: 0000000000003e9f RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:43:21 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:43:21 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:43:21 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:43:21 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:43:21 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:43:21 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:43:21 oleg133-client kernel: Call Trace: Aug 15 06:43:21 oleg133-client kernel: Aug 15 06:43:21 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:43:21 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:43:21 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:43:21 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:43:21 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:43:21 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:43:21 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:43:21 oleg133-client kernel: Aug 15 06:43:21 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:43:21 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:43:21 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:43:21 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:43:21 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:43:21 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:43:21 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:43:21 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:43:21 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:43:21 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:43:21 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:43:21 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:43:21 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:43:21 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:43:21 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:43:21 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:43:21 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:43:21 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:43:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:43:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:43:24 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:43:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:43:24 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:43:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:43:24 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:43:24 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:43:24 oleg133-client kernel: RAX: 0000000000005884 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:43:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:43:24 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:43:24 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:43:24 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:43:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:43:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:43:24 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:43:24 oleg133-client kernel: Call Trace: Aug 15 06:43:24 oleg133-client kernel: Aug 15 06:43:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:43:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:43:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:43:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:43:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:43:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:43:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:43:24 oleg133-client kernel: Aug 15 06:43:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:43:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:43:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:43:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:43:24 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:43:24 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:43:24 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:43:24 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:43:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:43:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:43:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:43:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:43:34 oleg133-client kernel: rcu: #0111-....: (625560 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=181826 Aug 15 06:43:34 oleg133-client kernel: #011(t=960028 jiffies g=278549 q=6082) Aug 15 06:43:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:43:34 oleg133-client kernel: INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 8.316 msecs Aug 15 06:43:34 oleg133-client kernel: NMI backtrace for cpu 0 Aug 15 06:43:34 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:43:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:43:34 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:43:34 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:43:34 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 Aug 15 06:43:34 oleg133-client kernel: RAX: 0000000000006713 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:43:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:43:34 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:43:34 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:43:34 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:43:34 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:43:34 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:43:34 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:43:34 oleg133-client kernel: Call Trace: Aug 15 06:43:34 oleg133-client kernel: Aug 15 06:43:34 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:43:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x98/0xa2 Aug 15 06:43:34 oleg133-client kernel: ? nmi_cpu_backtrace_handler+0x15/0x40 Aug 15 06:43:34 oleg133-client kernel: ? nmi_handle+0xbd/0x210 Aug 15 06:43:34 oleg133-client kernel: ? default_do_nmi+0x4b/0x1b0 Aug 15 06:43:34 oleg133-client kernel: ? do_nmi+0x280/0x380 Aug 15 06:43:34 oleg133-client kernel: ? end_repeat_nmi+0x16/0x1f Aug 15 06:43:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:43:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:43:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:43:34 oleg133-client kernel: Aug 15 06:43:34 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:43:34 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:43:34 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:43:34 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:43:34 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:43:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:43:34 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:43:34 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:43:34 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:43:34 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:43:34 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:43:34 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:43:34 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:43:34 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:43:34 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:43:34 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:43:34 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:43:34 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:43:34 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:43:34 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:43:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:43:34 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:43:34 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:43:34 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:43:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:43:35 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:43:35 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:43:35 oleg133-client kernel: Call Trace: Aug 15 06:43:35 oleg133-client kernel: Aug 15 06:43:35 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:43:35 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:43:35 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:43:35 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:43:35 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:43:35 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:43:35 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:43:35 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:43:35 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:43:35 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:43:35 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:43:35 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:43:35 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:43:35 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:43:35 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:43:35 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:43:35 oleg133-client kernel: Aug 15 06:43:35 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:43:35 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:43:35 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:43:35 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:43:35 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:43:35 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:43:35 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:43:35 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:43:35 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:43:35 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:43:35 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:43:35 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:43:35 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:43:35 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:43:35 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:43:35 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:43:35 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:43:35 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:43:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:43:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:43:52 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:43:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:43:52 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:43:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:43:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:43:52 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:43:52 oleg133-client kernel: RAX: 000000000000288d RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:43:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:43:52 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:43:52 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:43:52 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:43:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:43:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:43:52 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:43:52 oleg133-client kernel: Call Trace: Aug 15 06:43:52 oleg133-client kernel: Aug 15 06:43:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:43:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:43:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:43:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:43:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:43:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:43:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:43:52 oleg133-client kernel: Aug 15 06:43:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:43:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:43:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:43:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:43:52 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:43:52 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:43:52 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:43:52 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:43:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:43:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:43:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:44:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:44:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:44:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:44:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:44:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:44:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:44:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:44:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:44:00 oleg133-client kernel: RAX: 0000000000007c75 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:44:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:44:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:44:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:44:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:44:00 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:44:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:44:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:44:00 oleg133-client kernel: RDX: 0000000000000a0b RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:44:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:44:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:44:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:44:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:44:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:44:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:44:00 oleg133-client kernel: Call Trace: Aug 15 06:44:00 oleg133-client kernel: Aug 15 06:44:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:44:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:44:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:44:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:44:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:44:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:44:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:44:00 oleg133-client kernel: Aug 15 06:44:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:44:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:44:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:44:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:44:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:44:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:44:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:44:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:44:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:44:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:44:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:44:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:44:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:44:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:44:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:44:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:44:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:44:00 oleg133-client kernel: Call Trace: Aug 15 06:44:00 oleg133-client kernel: Aug 15 06:44:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:44:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:44:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:44:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:44:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:44:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:44:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:44:00 oleg133-client kernel: Aug 15 06:44:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:44:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:44:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:44:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:44:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:44:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:44:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:44:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:44:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:44:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:44:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:44:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:44:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:44:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:44:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:44:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:44:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:44:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:44:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:44:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:44:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:44:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:44:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:44:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:44:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:44:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:44:01 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:44:01 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:44:01 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:44:01 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:44:01 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:44:01 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:44:01 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:44:01 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:44:01 oleg133-client anacron[12030]: Job `cron.daily' started Aug 15 06:44:01 oleg133-client run-parts[36986]: (/etc/cron.daily) starting logrotate Aug 15 06:44:01 oleg133-client run-parts[36986]: (/etc/cron.daily) finished logrotate Aug 15 06:44:01 oleg133-client anacron[12030]: Job `cron.daily' terminated Aug 15 06:44:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:44:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:44:20 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:44:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:44:20 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:44:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:44:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:44:20 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 Aug 15 06:44:20 oleg133-client kernel: RAX: 000000000000738b RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:44:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:44:20 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:44:20 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:44:20 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:44:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:44:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:44:20 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:44:20 oleg133-client kernel: Call Trace: Aug 15 06:44:20 oleg133-client kernel: Aug 15 06:44:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:44:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:44:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:44:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:44:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:44:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:44:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:44:20 oleg133-client kernel: Aug 15 06:44:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:44:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:44:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:44:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:44:20 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:44:20 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:44:20 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:44:20 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:44:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:44:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:44:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:44:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:44:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:44:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:44:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:44:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:44:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:44:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:44:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:44:28 oleg133-client kernel: RAX: 0000000000004414 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:44:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:44:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:44:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:44:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:44:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:44:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:44:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:44:28 oleg133-client kernel: Call Trace: Aug 15 06:44:28 oleg133-client kernel: Aug 15 06:44:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:44:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:44:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:44:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:44:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:44:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:44:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:44:28 oleg133-client kernel: Aug 15 06:44:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:44:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:44:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:44:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:44:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:44:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:44:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:44:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:44:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:44:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:44:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:44:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:44:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:44:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:44:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:44:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:44:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:44:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:44:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:44:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:44:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:44:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:44:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:44:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:44:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:44:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:44:29 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:44:29 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:44:29 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:44:29 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:44:29 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:44:29 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:44:29 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:44:29 oleg133-client kernel: RDX: 00000000000048d3 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:44:29 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:44:29 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:44:29 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:44:29 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:44:29 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:44:29 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:44:29 oleg133-client kernel: Call Trace: Aug 15 06:44:29 oleg133-client kernel: Aug 15 06:44:29 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:44:29 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:44:29 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:44:29 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:44:29 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:44:29 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:44:29 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:44:29 oleg133-client kernel: Aug 15 06:44:29 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:44:29 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:44:29 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:44:29 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:44:29 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:44:29 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:44:29 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:44:29 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:44:29 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:44:29 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:44:29 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:44:29 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:44:29 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:44:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:44:29 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:44:29 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:44:29 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:44:29 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:44:29 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:44:48 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:44:48 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:44:48 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:44:48 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:44:48 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:44:48 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:44:48 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:44:48 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:44:48 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:44:48 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:44:48 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:44:48 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:44:48 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:44:48 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:44:48 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:44:48 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:44:48 oleg133-client kernel: Call Trace: Aug 15 06:44:48 oleg133-client kernel: Aug 15 06:44:48 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:44:48 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:44:48 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:44:48 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:44:48 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:44:48 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:44:48 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:44:48 oleg133-client kernel: Aug 15 06:44:48 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:44:48 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:44:48 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:44:48 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:44:48 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:44:48 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:44:48 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:44:48 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:44:48 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:44:48 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:44:48 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:44:48 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:44:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:44:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:44:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:44:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:44:56 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:44:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:44:56 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:44:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:44:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:44:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:44:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:44:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:44:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:44:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:44:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:44:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:44:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:44:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:44:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:44:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:44:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:44:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:44:56 oleg133-client kernel: Call Trace: Aug 15 06:44:56 oleg133-client kernel: Aug 15 06:44:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:44:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:44:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:44:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:44:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:44:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:44:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:44:56 oleg133-client kernel: Aug 15 06:44:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:44:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:44:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:44:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:44:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:44:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:44:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:44:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:44:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:44:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:44:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:44:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:44:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:44:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:44:56 oleg133-client kernel: Call Trace: Aug 15 06:44:56 oleg133-client kernel: Aug 15 06:44:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:44:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:44:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:44:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:44:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:44:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:44:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:44:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:44:56 oleg133-client kernel: Aug 15 06:44:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:44:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:44:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:44:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:44:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:44:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:44:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:44:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:44:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:44:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:44:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:44:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:44:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:44:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:44:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:44:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:44:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:44:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:44:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:44:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:44:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:44:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:44:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:44:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:44:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:44:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:44:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:44:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:44:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:44:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:45:16 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:45:16 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:45:16 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:45:16 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:45:16 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:45:16 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:45:16 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:45:16 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:45:16 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:45:16 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:45:16 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:45:16 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:45:16 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:45:16 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:45:16 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:45:16 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:45:16 oleg133-client kernel: Call Trace: Aug 15 06:45:16 oleg133-client kernel: Aug 15 06:45:16 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:45:16 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:45:16 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:45:16 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:45:16 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:45:16 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:45:16 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:45:16 oleg133-client kernel: Aug 15 06:45:16 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:45:16 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:45:16 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:45:16 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:45:16 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:45:16 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:45:16 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:45:16 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:45:16 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:45:16 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:45:16 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:45:16 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:45:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:45:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:45:24 oleg133-client kernel: Modules linked in: Aug 15 06:45:24 oleg133-client kernel: Modules linked in: Aug 15 06:45:24 oleg133-client kernel: lustre(O) osp(O) Aug 15 06:45:24 oleg133-client kernel: lustre(O) Aug 15 06:45:24 oleg133-client kernel: ofd(O) Aug 15 06:45:24 oleg133-client kernel: osp(O) Aug 15 06:45:24 oleg133-client kernel: lod(O) Aug 15 06:45:24 oleg133-client kernel: ofd(O) Aug 15 06:45:24 oleg133-client kernel: mdt(O) mdd(O) Aug 15 06:45:24 oleg133-client kernel: lod(O) Aug 15 06:45:24 oleg133-client kernel: mgs(O) Aug 15 06:45:24 oleg133-client kernel: mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:45:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:45:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:45:24 oleg133-client kernel: lquota(O) Aug 15 06:45:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:45:24 oleg133-client kernel: lfsck(O) Aug 15 06:45:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:45:24 oleg133-client kernel: obdecho(O) Aug 15 06:45:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 Aug 15 06:45:24 oleg133-client kernel: mgc(O) Aug 15 06:45:24 oleg133-client kernel: ORIG_RAX: ffffffffffffff13 Aug 15 06:45:24 oleg133-client kernel: mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 Aug 15 06:45:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:45:24 oleg133-client kernel: sha512_generic Aug 15 06:45:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:45:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:45:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:45:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:45:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:45:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:45:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:45:24 oleg133-client kernel: Call Trace: Aug 15 06:45:24 oleg133-client kernel: Aug 15 06:45:24 oleg133-client kernel: Aug 15 06:45:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:45:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:45:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:45:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:45:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:45:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:45:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:45:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:45:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 Aug 15 06:45:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:45:24 oleg133-client kernel: ORIG_RAX: ffffffffffffff13 Aug 15 06:45:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:45:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:45:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:45:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:45:24 oleg133-client kernel: Aug 15 06:45:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:45:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:45:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:45:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:45:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:45:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:45:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:45:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:45:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:45:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:45:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:45:24 oleg133-client kernel: Call Trace: Aug 15 06:45:24 oleg133-client kernel: Aug 15 06:45:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:45:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:45:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:45:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:45:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:45:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:45:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:45:24 oleg133-client kernel: Aug 15 06:45:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:45:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:45:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:45:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:45:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:45:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:45:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:45:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:45:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:45:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:45:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:45:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:45:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:45:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:45:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:45:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:45:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:45:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:45:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:45:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:45:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:45:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:45:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:45:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:45:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:45:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:45:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:45:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:45:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:45:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:45:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:45:44 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:45:44 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:45:44 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:45:44 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:45:44 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:45:44 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:45:44 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:45:44 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:45:44 oleg133-client kernel: RAX: 0000000000007e8e RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:45:44 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:45:44 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:45:44 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:45:44 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:45:44 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:45:44 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:45:44 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:45:44 oleg133-client kernel: Call Trace: Aug 15 06:45:44 oleg133-client kernel: Aug 15 06:45:44 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:45:44 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:45:44 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:45:44 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:45:44 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:45:44 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:45:44 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:45:44 oleg133-client kernel: Aug 15 06:45:44 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:45:44 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:45:44 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:45:44 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:45:44 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:45:44 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:45:44 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:45:44 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:45:44 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:45:44 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:45:44 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:45:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:45:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:45:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:45:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:45:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:45:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:45:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:45:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:45:52 oleg133-client kernel: RAX: 0000000000007c84 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:45:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:45:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:45:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:45:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:45:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:45:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:45:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:45:52 oleg133-client kernel: Call Trace: Aug 15 06:45:52 oleg133-client kernel: Aug 15 06:45:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:45:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:45:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:45:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:45:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:45:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:45:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) Aug 15 06:45:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:45:52 oleg133-client kernel: Aug 15 06:45:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:45:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:45:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:45:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:45:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:45:52 oleg133-client kernel: lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:45:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:45:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:45:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:45:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:45:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:45:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:45:52 oleg133-client kernel: RDX: 0000000000005791 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:45:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:45:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:45:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:45:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:45:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:45:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:45:52 oleg133-client kernel: Call Trace: Aug 15 06:45:52 oleg133-client kernel: Aug 15 06:45:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:45:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:45:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:45:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:45:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:45:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:45:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:45:52 oleg133-client kernel: Aug 15 06:45:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:45:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:45:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:45:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:45:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:45:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:45:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:45:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:45:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:45:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:45:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:45:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:45:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:45:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:45:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:45:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:45:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:45:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:45:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:45:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:45:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:45:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:45:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:45:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:45:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:45:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:45:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:45:53 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:45:53 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:45:53 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:45:53 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:45:53 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:45:53 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:45:53 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:45:53 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:45:53 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:45:53 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:45:53 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:45:53 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:46:12 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:46:12 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:46:12 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:46:12 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:46:12 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:46:12 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:46:12 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:46:12 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 Aug 15 06:46:12 oleg133-client kernel: RAX: 00000000000077a8 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:46:12 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:46:12 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:46:12 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:46:12 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:46:12 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:46:12 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:46:12 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:46:12 oleg133-client kernel: Call Trace: Aug 15 06:46:12 oleg133-client kernel: Aug 15 06:46:12 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:46:12 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:46:12 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:46:12 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:46:12 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:46:12 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:46:12 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:46:12 oleg133-client kernel: Aug 15 06:46:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:46:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:46:12 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:46:12 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:46:12 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:46:12 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:46:12 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:46:12 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:46:12 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:46:12 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:46:12 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:46:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:46:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:46:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:46:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:46:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:46:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:46:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:46:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:46:20 oleg133-client kernel: RAX: 000000000000337a RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:46:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:46:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:46:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:46:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:46:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:46:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:46:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:46:20 oleg133-client kernel: Call Trace: Aug 15 06:46:20 oleg133-client kernel: Aug 15 06:46:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:46:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:46:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:46:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:46:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:46:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:46:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:46:20 oleg133-client kernel: Aug 15 06:46:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:46:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:46:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:46:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:46:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:46:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:46:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:46:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:46:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:46:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:46:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:46:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:46:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:46:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:46:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:46:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:46:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:46:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:46:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:46:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:46:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:46:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:46:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:46:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:46:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:46:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:46:21 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:46:21 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:46:21 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:46:21 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:46:21 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:46:21 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:46:21 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:46:21 oleg133-client kernel: RDX: 00000000000009a0 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:46:21 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:46:21 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:46:21 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:46:21 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:46:21 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:46:21 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:46:21 oleg133-client kernel: Call Trace: Aug 15 06:46:21 oleg133-client kernel: Aug 15 06:46:21 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:46:21 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:46:21 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:46:21 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:46:21 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:46:21 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:46:21 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:46:21 oleg133-client kernel: Aug 15 06:46:21 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:46:21 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:46:21 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:46:21 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:46:21 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:46:21 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:46:21 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:46:21 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:46:21 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:46:21 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:46:21 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:46:21 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:46:21 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:46:21 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:46:21 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:46:21 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:46:21 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:46:21 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:46:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:46:34 oleg133-client kernel: rcu: #0111-....: (735274 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=215128 Aug 15 06:46:34 oleg133-client kernel: #011(t=1140032 jiffies g=278549 q=7370) Aug 15 06:46:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:46:34 oleg133-client kernel: NMI backtrace for cpu 0 Aug 15 06:46:34 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:46:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:46:34 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:46:34 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:46:34 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 Aug 15 06:46:34 oleg133-client kernel: RAX: 0000000000001394 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:46:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:46:34 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:46:34 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:46:34 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:46:34 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:46:34 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:46:34 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:46:34 oleg133-client kernel: Call Trace: Aug 15 06:46:34 oleg133-client kernel: Aug 15 06:46:34 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:46:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x98/0xa2 Aug 15 06:46:34 oleg133-client kernel: ? nmi_cpu_backtrace_handler+0x15/0x40 Aug 15 06:46:34 oleg133-client kernel: ? nmi_handle+0xbd/0x210 Aug 15 06:46:34 oleg133-client kernel: ? default_do_nmi+0x4b/0x1b0 Aug 15 06:46:34 oleg133-client kernel: ? do_nmi+0x280/0x380 Aug 15 06:46:34 oleg133-client kernel: ? end_repeat_nmi+0x16/0x1f Aug 15 06:46:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:46:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:46:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:46:34 oleg133-client kernel: Aug 15 06:46:34 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:46:34 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:46:34 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:46:34 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:46:34 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:46:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:46:34 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:46:34 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:46:34 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:46:34 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:46:34 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:46:34 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:46:34 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:46:34 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:46:34 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:46:34 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:46:34 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:46:34 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:46:34 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:46:34 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:46:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:46:34 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:46:34 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:46:34 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:46:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:46:35 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:46:35 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:46:35 oleg133-client kernel: Call Trace: Aug 15 06:46:35 oleg133-client kernel: Aug 15 06:46:35 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:46:35 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:46:35 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:46:35 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:46:35 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:46:35 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:46:35 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:46:35 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:46:35 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:46:35 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:46:35 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:46:35 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:46:35 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:46:35 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:46:35 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:46:35 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:46:35 oleg133-client kernel: Aug 15 06:46:35 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:46:35 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:46:35 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:46:35 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:46:35 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:46:35 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:46:35 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:46:35 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:46:35 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:46:35 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:46:35 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:46:35 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:46:35 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:46:35 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:46:35 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:46:35 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:46:35 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:46:35 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:46:40 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:46:40 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:46:40 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:46:40 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:46:40 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:46:40 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:46:40 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:46:40 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:46:40 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:46:40 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:46:40 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:46:40 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:46:40 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:46:40 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:46:40 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:46:40 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:46:40 oleg133-client kernel: Call Trace: Aug 15 06:46:40 oleg133-client kernel: Aug 15 06:46:40 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:46:40 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:46:40 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:46:40 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:46:40 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:46:40 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:46:40 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:46:40 oleg133-client kernel: Aug 15 06:46:40 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:46:40 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:46:40 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:46:40 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:46:40 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:46:40 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:46:40 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:46:40 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:46:40 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:46:40 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:46:40 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:46:40 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:47:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:47:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:47:00 oleg133-client kernel: Modules linked in: Aug 15 06:47:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix Aug 15 06:47:00 oleg133-client kernel: lustre(O) Aug 15 06:47:00 oleg133-client kernel: serio_raw libata dm_mirror dm_region_hash Aug 15 06:47:00 oleg133-client kernel: osp(O) Aug 15 06:47:00 oleg133-client kernel: dm_log Aug 15 06:47:00 oleg133-client kernel: ofd(O) Aug 15 06:47:00 oleg133-client kernel: dm_mod sha512_ssse3 sha512_generic Aug 15 06:47:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:47:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:47:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:47:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:47:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:47:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:47:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:47:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:47:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:47:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:47:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:47:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:47:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:47:00 oleg133-client kernel: Call Trace: Aug 15 06:47:00 oleg133-client kernel: Aug 15 06:47:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:47:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:47:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:47:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:47:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:47:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:47:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:47:00 oleg133-client kernel: Aug 15 06:47:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:47:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:47:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:47:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:47:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:47:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:47:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:47:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:47:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:47:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:47:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:47:00 oleg133-client kernel: lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:47:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:47:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:47:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:47:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:47:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:47:00 oleg133-client kernel: RAX: 0000000000007b10 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:47:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:47:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:47:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:47:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:47:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:47:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:47:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:47:00 oleg133-client kernel: Call Trace: Aug 15 06:47:00 oleg133-client kernel: Aug 15 06:47:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:47:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:47:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:47:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:47:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:47:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:47:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:47:00 oleg133-client kernel: Aug 15 06:47:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:47:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:47:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:47:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:47:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:47:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:47:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:47:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:47:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:47:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:47:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:47:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:47:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:47:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:47:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:47:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:47:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:47:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:47:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:47:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:47:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:47:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:47:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:47:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:47:08 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:47:08 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:47:08 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:47:08 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:47:08 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:47:08 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:47:08 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:47:08 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:47:08 oleg133-client kernel: RAX: 0000000000006a4d RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:47:08 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:47:08 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:47:08 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:47:08 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:47:08 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:47:08 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:47:08 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:47:08 oleg133-client kernel: Call Trace: Aug 15 06:47:08 oleg133-client kernel: Aug 15 06:47:08 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:47:08 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:47:08 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:47:08 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:47:08 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:47:08 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:47:08 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:47:08 oleg133-client kernel: Aug 15 06:47:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:47:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:47:08 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:47:08 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:47:08 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:47:08 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:47:08 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:47:08 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:47:08 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:47:08 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:47:08 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:47:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:47:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:47:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:47:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:47:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:47:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:47:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:47:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:47:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:47:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:47:28 oleg133-client kernel: RDX: 0000000000003cc3 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:47:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:47:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:47:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:47:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:47:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:47:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:47:28 oleg133-client kernel: Call Trace: Aug 15 06:47:28 oleg133-client kernel: Aug 15 06:47:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:47:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:47:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:47:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:47:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:47:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:47:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:47:28 oleg133-client kernel: Aug 15 06:47:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:47:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:47:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:47:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:47:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:47:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:47:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:47:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:47:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:47:28 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 Aug 15 06:47:28 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:47:28 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:47:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:47:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:47:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:47:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:47:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:47:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:47:28 oleg133-client kernel: Call Trace: Aug 15 06:47:28 oleg133-client kernel: Aug 15 06:47:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:47:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:47:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:47:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:47:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:47:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:47:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:47:28 oleg133-client kernel: Aug 15 06:47:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:47:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:47:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:47:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:47:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:47:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:47:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:47:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:47:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:47:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:47:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:47:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:47:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:47:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:47:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:47:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:47:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:47:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:47:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:47:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:47:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:47:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:47:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:47:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:47:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:47:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:47:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:47:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:47:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:47:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:47:36 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:47:36 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:47:36 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:47:36 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:47:36 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:47:36 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:47:36 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:47:36 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:47:36 oleg133-client kernel: RAX: 0000000000007d8d RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:47:36 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:47:36 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:47:36 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:47:36 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:47:36 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:47:36 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:47:36 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:47:36 oleg133-client kernel: Call Trace: Aug 15 06:47:36 oleg133-client kernel: Aug 15 06:47:36 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:47:36 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:47:36 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:47:36 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:47:36 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:47:36 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:47:36 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:47:36 oleg133-client kernel: Aug 15 06:47:36 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:47:36 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:47:36 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:47:36 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:47:36 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:47:36 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:47:36 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:47:36 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:47:36 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:47:36 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:47:36 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:47:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:47:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:47:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:47:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:47:56 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:47:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:47:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:47:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:47:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:47:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:47:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:47:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:47:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:47:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:47:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:47:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:47:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:47:56 oleg133-client kernel: Call Trace: Aug 15 06:47:56 oleg133-client kernel: Aug 15 06:47:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:47:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:47:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:47:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:47:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:47:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:47:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:47:56 oleg133-client kernel: Aug 15 06:47:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:47:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:47:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:47:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:47:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:47:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: osp(O) Aug 15 06:47:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:47:56 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:47:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:47:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:47:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:47:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:47:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:47:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:47:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:47:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:47:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:47:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:47:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:47:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:47:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:47:56 oleg133-client kernel: Call Trace: Aug 15 06:47:56 oleg133-client kernel: Aug 15 06:47:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:47:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:47:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:47:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:47:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:47:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:47:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:47:56 oleg133-client kernel: Aug 15 06:47:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:47:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:47:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:47:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:47:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:47:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:47:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:47:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:47:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:47:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:47:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:47:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:47:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:47:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:47:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:47:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:47:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:47:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:47:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:47:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:47:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:47:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:47:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:47:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:47:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:47:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:47:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:47:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:47:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:47:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:48:04 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:48:04 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:48:04 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:48:04 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:48:04 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:48:04 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:48:04 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:48:04 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:48:04 oleg133-client kernel: RAX: 0000000000006625 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:48:04 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:48:04 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:48:04 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:48:04 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:48:04 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:48:04 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:48:04 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:48:04 oleg133-client kernel: Call Trace: Aug 15 06:48:04 oleg133-client kernel: Aug 15 06:48:04 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:48:04 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:48:04 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:48:04 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:48:04 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:48:04 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:48:04 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:48:04 oleg133-client kernel: Aug 15 06:48:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:48:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:48:04 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:48:04 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:48:04 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:48:04 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:48:04 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:48:04 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:48:04 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:48:04 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:48:04 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:48:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:48:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:48:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:48:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:48:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:48:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:48:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:48:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:48:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:48:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:48:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:48:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:48:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:48:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:48:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:48:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:48:24 oleg133-client kernel: Call Trace: Aug 15 06:48:24 oleg133-client kernel: Aug 15 06:48:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:48:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:48:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:48:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:48:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:48:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:48:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:48:24 oleg133-client kernel: Aug 15 06:48:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:48:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:48:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:48:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:48:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:48:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:48:24 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:48:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:48:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:48:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:48:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:48:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:48:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:48:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:48:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:48:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:48:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:48:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:48:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:48:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:48:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:48:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:48:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:48:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:48:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:48:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:48:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:48:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:48:25 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:48:25 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:48:25 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:48:25 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:48:25 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:48:25 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:48:25 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:48:25 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:48:25 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:48:25 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:48:25 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:48:25 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:48:25 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:48:25 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:48:25 oleg133-client kernel: Call Trace: Aug 15 06:48:25 oleg133-client kernel: Aug 15 06:48:25 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:48:25 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:48:25 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:48:25 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:48:25 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:48:25 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:48:25 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:48:25 oleg133-client kernel: Aug 15 06:48:25 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:48:25 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:48:25 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:48:25 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:48:25 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:48:25 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:48:25 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:48:25 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:48:25 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:48:25 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:48:25 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:48:25 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:48:25 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:48:25 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:48:25 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:48:25 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:48:25 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:48:25 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:48:25 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:48:32 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:48:32 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:48:32 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:48:32 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:48:32 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:48:32 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:48:32 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:48:32 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:48:32 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:48:32 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:48:32 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:48:32 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:48:32 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:48:32 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:48:32 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:48:32 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:48:32 oleg133-client kernel: Call Trace: Aug 15 06:48:32 oleg133-client kernel: Aug 15 06:48:32 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:48:32 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:48:32 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:48:32 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:48:32 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:48:32 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:48:32 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:48:32 oleg133-client kernel: Aug 15 06:48:32 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:48:32 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:48:32 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:48:32 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:48:32 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:48:32 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:48:32 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:48:32 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:48:32 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:48:32 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:48:32 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:48:32 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:48:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:48:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:48:52 oleg133-client kernel: Modules linked in: Aug 15 06:48:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) Aug 15 06:48:52 oleg133-client kernel: lustre(O) osp(O) Aug 15 06:48:52 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:48:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:48:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:48:52 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:48:52 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:48:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:48:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:48:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:48:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:48:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:48:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:48:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:48:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:48:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:48:52 oleg133-client kernel: Call Trace: Aug 15 06:48:52 oleg133-client kernel: Aug 15 06:48:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:48:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:48:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:48:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:48:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:48:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:48:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:48:52 oleg133-client kernel: Aug 15 06:48:52 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:48:52 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:48:52 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:48:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:48:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:48:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:48:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:48:52 oleg133-client kernel: ofd(O) Aug 15 06:48:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:48:52 oleg133-client kernel: lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:48:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:48:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:48:52 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:48:52 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:48:52 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:48:52 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:48:52 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:48:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:48:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:48:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:48:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:48:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:48:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:48:52 oleg133-client kernel: Call Trace: Aug 15 06:48:52 oleg133-client kernel: Aug 15 06:48:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:48:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:48:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:48:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:48:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:48:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:48:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:48:52 oleg133-client kernel: Aug 15 06:48:52 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:48:52 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:48:52 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:48:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:48:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:48:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:48:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:48:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:48:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:48:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:48:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:48:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:48:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:48:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:48:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:48:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:48:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:48:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:48:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:48:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:48:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:48:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:48:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:48:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:48:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:48:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:48:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:48:53 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:48:53 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:48:53 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:48:53 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:48:53 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:48:53 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:48:53 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:48:53 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:48:53 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:48:53 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:48:53 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:49:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:49:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:49:00 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:49:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:49:00 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:49:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:49:00 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:49:00 oleg133-client kernel: RAX: 0000000000007df4 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:49:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:49:00 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:49:00 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:49:00 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:49:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:49:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:49:00 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:49:00 oleg133-client kernel: Call Trace: Aug 15 06:49:00 oleg133-client kernel: Aug 15 06:49:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:49:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:49:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:49:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:49:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:49:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:49:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:49:00 oleg133-client kernel: Aug 15 06:49:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:49:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:49:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:49:00 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:49:00 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:49:00 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:49:00 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:49:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:49:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:49:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:49:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:49:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:49:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:49:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:49:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) Aug 15 06:49:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:49:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:49:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:49:20 oleg133-client kernel: RAX: 000000000000401f RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:49:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:49:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:49:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:49:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:49:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:49:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:49:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:49:20 oleg133-client kernel: Call Trace: Aug 15 06:49:20 oleg133-client kernel: Aug 15 06:49:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:49:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:49:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:49:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:49:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:49:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:49:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:49:20 oleg133-client kernel: Aug 15 06:49:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:20 oleg133-client kernel: lov(O) Aug 15 06:49:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:49:20 oleg133-client kernel: osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common Aug 15 06:49:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:49:20 oleg133-client kernel: sb_edac Aug 15 06:49:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:49:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: rapl Aug 15 06:49:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:49:20 oleg133-client kernel: pcspkr Aug 15 06:49:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: i2c_piix4 Aug 15 06:49:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:49:20 oleg133-client kernel: squashfs Aug 15 06:49:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:49:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:49:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:49:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:49:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:49:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:49:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:49:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:49:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:49:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 Aug 15 06:49:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:49:20 oleg133-client kernel: ORIG_RAX: ffffffffffffff13 Aug 15 06:49:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:49:20 oleg133-client kernel: RDX: 0000000000000a2c RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:49:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:49:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:49:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:49:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:49:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:49:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:49:20 oleg133-client kernel: Call Trace: Aug 15 06:49:20 oleg133-client kernel: Aug 15 06:49:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:49:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:49:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:49:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:49:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:49:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:49:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:49:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:49:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:49:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:49:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:49:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:49:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:49:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:49:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:49:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:49:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:49:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:49:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:49:20 oleg133-client kernel: Aug 15 06:49:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:49:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:49:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:49:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:49:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:49:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:49:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:49:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:49:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:49:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:49:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:49:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:49:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:49:28 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:49:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:49:28 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:49:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:49:28 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:49:28 oleg133-client kernel: RAX: 0000000000002d99 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:49:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:49:28 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:49:28 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:49:28 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:49:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:49:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:49:28 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:49:28 oleg133-client kernel: Call Trace: Aug 15 06:49:28 oleg133-client kernel: Aug 15 06:49:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:49:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:49:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:49:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:49:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:49:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:49:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:49:28 oleg133-client kernel: Aug 15 06:49:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:49:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:49:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:49:28 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:49:28 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:49:28 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:49:28 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:49:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:49:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:49:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:49:28 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:49:28 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:49:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:49:34 oleg133-client kernel: rcu: #0111-....: (853636 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=249812 Aug 15 06:49:34 oleg133-client kernel: #011(t=1320035 jiffies g=278549 q=8202) Aug 15 06:49:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:49:34 oleg133-client kernel: NMI backtrace for cpu 0 Aug 15 06:49:34 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:49:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:49:34 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:34 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:49:34 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000212 Aug 15 06:49:34 oleg133-client kernel: RAX: 000000000000329b RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:49:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:49:34 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:49:34 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:49:34 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:49:34 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:49:34 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:49:34 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:49:34 oleg133-client kernel: Call Trace: Aug 15 06:49:34 oleg133-client kernel: Aug 15 06:49:34 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:49:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x98/0xa2 Aug 15 06:49:34 oleg133-client kernel: ? nmi_cpu_backtrace_handler+0x15/0x40 Aug 15 06:49:34 oleg133-client kernel: ? nmi_handle+0xbd/0x210 Aug 15 06:49:34 oleg133-client kernel: ? default_do_nmi+0x4b/0x1b0 Aug 15 06:49:34 oleg133-client kernel: ? do_nmi+0x280/0x380 Aug 15 06:49:34 oleg133-client kernel: ? end_repeat_nmi+0x16/0x1f Aug 15 06:49:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:34 oleg133-client kernel: Aug 15 06:49:34 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:49:34 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:49:34 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:49:34 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:49:34 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:49:34 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:49:34 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:49:34 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:49:34 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:49:34 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:49:34 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:49:34 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:49:34 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:49:34 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:49:34 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:49:34 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:49:34 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:49:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:49:34 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:49:34 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:49:34 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:49:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:49:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:49:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:49:34 oleg133-client kernel: Call Trace: Aug 15 06:49:34 oleg133-client kernel: Aug 15 06:49:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:49:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:49:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:49:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:49:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:49:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:49:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:49:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:49:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:49:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:49:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:49:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:49:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:49:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:49:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:49:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:49:34 oleg133-client kernel: Aug 15 06:49:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x185/0x6d0 Aug 15 06:49:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:49:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:49:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:49:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:49:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:49:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:49:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:49:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:49:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:49:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:49:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:49:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:49:56 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:49:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:49:56 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:49:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:56 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:49:56 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:49:56 oleg133-client kernel: RAX: 000000000000152f RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:49:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:49:56 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:49:56 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:49:56 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:49:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:49:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:49:56 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:49:56 oleg133-client kernel: Call Trace: Aug 15 06:49:56 oleg133-client kernel: Aug 15 06:49:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:49:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:49:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:49:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:49:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:49:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:49:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:49:56 oleg133-client kernel: Aug 15 06:49:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:49:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:49:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:49:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:49:56 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:49:56 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:49:56 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:49:56 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:49:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:49:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:49:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:50:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:50:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:50:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:50:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:50:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:50:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:50:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:50:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:50:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:50:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:50:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:50:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:50:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:50:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:50:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:50:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:50:00 oleg133-client kernel: Call Trace: Aug 15 06:50:00 oleg133-client kernel: Aug 15 06:50:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:50:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:50:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:50:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:50:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:50:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:50:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:50:00 oleg133-client kernel: Aug 15 06:50:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:50:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:50:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:50:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:50:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:50:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:50:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:50:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:50:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:50:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:50:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:50:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:50:00 oleg133-client kernel: RAX: 000000000000650c RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:50:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:50:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:50:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:50:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:50:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:50:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:50:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:50:00 oleg133-client kernel: Call Trace: Aug 15 06:50:00 oleg133-client kernel: Aug 15 06:50:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:50:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:50:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:50:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:50:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:50:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:50:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:50:00 oleg133-client kernel: Aug 15 06:50:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:50:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:50:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:50:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:50:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:50:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:50:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:50:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:50:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:50:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:50:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:50:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:50:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:50:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:50:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:50:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:50:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:50:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:50:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:50:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:50:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:50:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:50:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:50:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:50:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:50:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:50:01 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:50:01 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:50:01 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:50:01 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:50:01 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:50:01 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:50:01 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:50:01 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:50:01 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:50:01 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:50:01 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:50:01 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:50:01 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:50:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:50:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:50:24 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:50:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:50:24 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:50:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:50:24 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:50:24 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:50:24 oleg133-client kernel: RAX: 0000000000007f0f RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:50:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:50:24 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:50:24 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:50:24 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:50:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:50:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:50:24 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:50:24 oleg133-client kernel: Call Trace: Aug 15 06:50:24 oleg133-client kernel: Aug 15 06:50:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:50:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:50:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:50:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:50:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:50:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:50:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:50:24 oleg133-client kernel: Aug 15 06:50:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:50:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:50:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:50:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:50:24 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:50:24 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:50:24 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:50:24 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:50:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:50:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:50:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:50:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:50:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:50:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:50:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:50:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:50:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:50:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:50:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:50:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:50:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:50:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:50:28 oleg133-client kernel: RDX: 0000000000001ed8 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:50:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:50:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:50:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:50:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:50:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:50:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:50:28 oleg133-client kernel: Call Trace: Aug 15 06:50:28 oleg133-client kernel: Aug 15 06:50:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:50:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:50:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:50:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:50:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:50:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:50:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:50:28 oleg133-client kernel: Aug 15 06:50:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:50:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:50:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:50:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:50:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:50:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:50:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:50:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:50:28 oleg133-client kernel: RAX: 0000000000006aa9 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:50:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:50:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:50:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:50:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:50:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:50:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:50:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:50:28 oleg133-client kernel: Call Trace: Aug 15 06:50:28 oleg133-client kernel: Aug 15 06:50:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:50:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:50:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:50:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:50:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:50:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:50:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:50:28 oleg133-client kernel: Aug 15 06:50:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:50:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:50:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:50:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:50:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:50:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:50:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:50:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:50:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:50:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:50:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:50:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:50:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:50:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:50:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:50:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:50:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:50:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:50:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:50:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:50:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:50:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:50:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:50:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:50:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:50:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:50:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:50:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:50:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:50:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:50:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:50:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:50:52 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:50:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:50:52 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:50:52 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:50:52 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:50:52 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:50:52 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:50:52 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:50:52 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:50:52 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:50:52 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:50:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:50:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:50:52 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:50:52 oleg133-client kernel: Call Trace: Aug 15 06:50:52 oleg133-client kernel: Aug 15 06:50:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:50:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:50:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:50:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:50:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:50:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:50:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:50:52 oleg133-client kernel: Aug 15 06:50:52 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:50:52 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:50:52 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:50:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:50:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:50:52 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:50:52 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:50:52 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:50:52 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:50:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:50:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:50:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:50:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:50:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:50:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:50:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:50:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:50:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x17c/0x6d0 Aug 15 06:50:56 oleg133-client kernel: Code: 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 48 83 05 5c d6 dc 02 01 <83> ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 75 e6 48 83 05 2c d6 dc Aug 15 06:50:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:50:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:50:56 oleg133-client kernel: RDX: 0000000000004b07 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:50:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:50:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:50:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:50:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:50:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:50:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:50:56 oleg133-client kernel: Call Trace: Aug 15 06:50:56 oleg133-client kernel: Aug 15 06:50:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:50:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:50:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:50:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:50:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:50:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:50:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:50:56 oleg133-client kernel: Aug 15 06:50:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x17c/0x6d0 Aug 15 06:50:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:50:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:50:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:50:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:50:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:50:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:50:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:50:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:50:56 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:50:56 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:50:56 oleg133-client kernel: RAX: 00000000000079e1 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:50:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:50:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:50:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:50:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:50:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:50:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:50:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:50:56 oleg133-client kernel: Call Trace: Aug 15 06:50:56 oleg133-client kernel: Aug 15 06:50:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:50:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:50:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:50:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:50:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:50:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:50:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:50:56 oleg133-client kernel: Aug 15 06:50:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:50:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:50:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:50:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:50:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:50:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:50:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:50:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:50:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:50:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:50:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:50:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:50:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:50:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:50:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:50:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:50:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:50:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:50:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:50:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:50:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:50:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:50:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:50:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:50:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:50:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:50:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:50:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:50:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:51:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:51:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:51:20 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:51:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:51:20 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:51:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:51:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:51:20 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:51:20 oleg133-client kernel: RAX: 0000000000001a1d RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:51:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:51:20 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:51:20 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:51:20 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:51:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:51:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:51:20 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:51:20 oleg133-client kernel: Call Trace: Aug 15 06:51:20 oleg133-client kernel: Aug 15 06:51:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:51:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:51:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:51:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:51:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:51:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:51:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:51:20 oleg133-client kernel: Aug 15 06:51:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:51:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:51:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:51:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:51:20 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:51:20 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:51:20 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:51:20 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:51:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:51:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:51:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:51:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:51:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:51:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) Aug 15 06:51:24 oleg133-client kernel: Modules linked in: Aug 15 06:51:24 oleg133-client kernel: lod(O) mdt(O) mdd(O) mgs(O) Aug 15 06:51:24 oleg133-client kernel: lustre(O) Aug 15 06:51:24 oleg133-client kernel: lquota(O) Aug 15 06:51:24 oleg133-client kernel: osp(O) Aug 15 06:51:24 oleg133-client kernel: lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:51:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:51:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:51:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:51:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:51:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:51:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:51:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:51:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:51:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:51:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:51:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:51:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:51:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:51:24 oleg133-client kernel: Call Trace: Aug 15 06:51:24 oleg133-client kernel: Aug 15 06:51:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:51:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:51:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:51:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:51:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:51:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:51:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:51:24 oleg133-client kernel: Aug 15 06:51:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:51:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:51:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:51:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:51:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:51:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:51:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: ofd(O) Aug 15 06:51:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:51:24 oleg133-client kernel: lod(O) Aug 15 06:51:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: mdt(O) Aug 15 06:51:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:51:24 oleg133-client kernel: mdd(O) Aug 15 06:51:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:51:24 oleg133-client kernel: mgs(O) Aug 15 06:51:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:51:24 oleg133-client kernel: lquota(O) Aug 15 06:51:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:51:24 oleg133-client kernel: lfsck(O) Aug 15 06:51:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:51:24 oleg133-client kernel: obdecho(O) Aug 15 06:51:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:51:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:51:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:51:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:51:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:51:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:51:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:51:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:51:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:51:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:51:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:51:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:51:24 oleg133-client kernel: mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:51:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:51:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:51:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:51:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:51:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:51:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:51:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:51:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:51:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:51:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:51:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:51:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:51:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:51:24 oleg133-client kernel: Call Trace: Aug 15 06:51:24 oleg133-client kernel: Aug 15 06:51:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:51:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:51:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:51:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:51:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:51:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:51:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:51:24 oleg133-client kernel: Aug 15 06:51:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:51:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:51:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:51:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:51:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:51:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:51:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:51:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:51:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:51:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:51:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:51:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:51:48 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:51:48 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:51:48 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:51:48 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:51:48 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:51:48 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:51:48 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:51:48 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:51:48 oleg133-client kernel: RAX: 0000000000006b0c RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:51:48 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:51:48 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:51:48 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:51:48 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:51:48 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:51:48 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:51:48 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:51:48 oleg133-client kernel: Call Trace: Aug 15 06:51:48 oleg133-client kernel: Aug 15 06:51:48 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:51:48 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:51:48 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:51:48 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:51:48 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:51:48 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:51:48 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:51:48 oleg133-client kernel: Aug 15 06:51:48 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:51:48 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:51:48 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:51:48 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:51:48 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:51:48 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:51:48 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:51:48 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:51:48 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:51:48 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:51:48 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:51:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:51:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:51:52 oleg133-client kernel: Modules linked in: Aug 15 06:51:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:51:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:51:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:51:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:51:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:51:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:51:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:51:52 oleg133-client kernel: RDX: 00000000000032bd RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:51:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:51:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:51:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:51:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:51:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:51:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:51:52 oleg133-client kernel: Call Trace: Aug 15 06:51:52 oleg133-client kernel: Aug 15 06:51:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:51:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:51:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:51:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:51:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:51:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:51:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:51:52 oleg133-client kernel: Aug 15 06:51:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:51:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:51:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:51:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:51:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:51:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:51:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:51:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:51:52 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:51:52 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:51:52 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:51:52 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:51:52 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:51:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:51:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:51:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:51:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:51:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:51:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:51:52 oleg133-client kernel: Call Trace: Aug 15 06:51:52 oleg133-client kernel: Aug 15 06:51:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:51:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:51:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:51:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:51:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:51:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:51:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:51:52 oleg133-client kernel: Aug 15 06:51:52 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:51:52 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:51:52 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:51:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:51:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:51:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:51:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:51:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:51:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:51:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:51:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:51:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:51:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:51:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:51:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:51:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:51:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:51:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:51:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:51:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:51:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:51:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:51:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:51:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:51:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:51:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:51:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:51:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:51:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:51:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:52:16 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:52:16 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:52:16 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:52:16 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:52:16 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:52:16 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:52:16 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:52:16 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:52:16 oleg133-client kernel: RAX: 000000000000780b RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:52:16 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:52:16 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:52:16 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:52:16 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:52:16 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:52:16 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:52:16 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:52:16 oleg133-client kernel: Call Trace: Aug 15 06:52:16 oleg133-client kernel: Aug 15 06:52:16 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:52:16 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:52:16 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:52:16 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:52:16 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:52:16 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:52:16 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:52:16 oleg133-client kernel: Aug 15 06:52:16 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:52:16 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:52:16 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:52:16 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:52:16 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:52:16 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:52:16 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:52:16 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:52:16 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:52:16 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:52:16 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:52:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:52:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:52:20 oleg133-client kernel: Modules linked in: Aug 15 06:52:20 oleg133-client kernel: Modules linked in: Aug 15 06:52:20 oleg133-client kernel: lustre(O) osp(O) Aug 15 06:52:20 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:52:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:52:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:52:20 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:52:20 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:52:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:52:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:52:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:52:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:52:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:52:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:52:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:52:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:52:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:52:20 oleg133-client kernel: Call Trace: Aug 15 06:52:20 oleg133-client kernel: Aug 15 06:52:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:52:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:52:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:52:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:52:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:52:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:52:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:52:20 oleg133-client kernel: Aug 15 06:52:20 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:52:20 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:52:20 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:52:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:52:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:52:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:52:20 oleg133-client kernel: ofd(O) Aug 15 06:52:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: lod(O) Aug 15 06:52:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:52:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:52:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:52:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:52:20 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:52:20 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:52:20 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:52:20 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:52:20 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:52:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:52:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:52:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:52:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:52:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:52:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:52:20 oleg133-client kernel: Call Trace: Aug 15 06:52:20 oleg133-client kernel: Aug 15 06:52:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:52:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:52:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:52:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:52:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:52:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:52:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:52:20 oleg133-client kernel: Aug 15 06:52:20 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:52:20 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:52:20 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:52:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:52:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:52:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:52:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:52:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:52:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:52:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:52:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:52:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:52:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:52:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:52:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:52:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:52:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:52:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:52:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:52:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:52:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:52:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:52:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:52:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:52:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:52:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:52:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:52:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:52:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:52:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:52:34 oleg133-client kernel: rcu: #0111-....: (977932 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=285099 Aug 15 06:52:34 oleg133-client kernel: #011(t=1500038 jiffies g=278549 q=8876) Aug 15 06:52:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:52:34 oleg133-client kernel: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 Aug 15 06:52:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:52:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:52:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:52:34 oleg133-client kernel: Call Trace: Aug 15 06:52:34 oleg133-client kernel: Aug 15 06:52:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:52:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:52:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:52:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:52:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:52:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:52:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:52:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:52:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:52:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:52:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:52:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:52:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:52:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:52:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:52:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:52:34 oleg133-client kernel: Aug 15 06:52:34 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:52:34 oleg133-client kernel: ? kvm_wait+0xcd/0xe0 Aug 15 06:52:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:52:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:52:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:52:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:52:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:52:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:52:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:52:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:52:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:52:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:52:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:52:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:52:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:52:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:52:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:52:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:52:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:52:44 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:52:44 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:52:44 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:52:44 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:52:44 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:52:44 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:52:44 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:52:44 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:52:44 oleg133-client kernel: RAX: 0000000000004f62 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:52:44 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:52:44 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:52:44 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:52:44 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:52:44 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:52:44 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:52:44 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:52:44 oleg133-client kernel: Call Trace: Aug 15 06:52:44 oleg133-client kernel: Aug 15 06:52:44 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:52:44 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:52:44 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:52:44 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:52:44 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:52:44 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:52:44 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:52:44 oleg133-client kernel: Aug 15 06:52:44 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:52:44 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:52:44 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:52:44 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:52:44 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:52:44 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:52:44 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:52:44 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:52:44 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:52:44 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:52:44 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:53:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:53:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:53:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:53:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:53:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:53:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:53:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:53:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:53:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:53:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:53:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:53:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:53:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:53:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:53:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:53:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:53:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:53:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:53:00 oleg133-client kernel: Call Trace: Aug 15 06:53:00 oleg133-client kernel: Aug 15 06:53:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:53:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:53:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:53:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:53:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:53:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:53:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:53:00 oleg133-client kernel: Aug 15 06:53:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:53:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:53:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:53:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:53:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:53:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:53:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:53:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:53:00 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:53:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:53:00 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:53:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:53:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:53:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:53:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:53:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:53:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:53:00 oleg133-client kernel: Call Trace: Aug 15 06:53:00 oleg133-client kernel: Aug 15 06:53:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:53:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:53:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:53:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:53:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:53:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:53:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:53:00 oleg133-client kernel: Aug 15 06:53:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:53:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:53:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:53:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:53:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:53:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:53:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:53:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:53:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:53:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:53:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:53:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:53:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:53:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:53:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:53:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:53:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:53:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:53:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:53:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:53:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:53:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:53:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:53:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:53:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:53:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:53:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:53:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:53:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:53:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:53:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:53:12 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:53:12 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:53:12 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:53:12 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:53:12 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:53:12 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:53:12 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:53:12 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:53:12 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:53:12 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:53:12 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:53:12 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:53:12 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:53:12 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:53:12 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:53:12 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:53:12 oleg133-client kernel: Call Trace: Aug 15 06:53:12 oleg133-client kernel: Aug 15 06:53:12 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:53:12 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:53:12 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:53:12 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:53:12 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:53:12 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:53:12 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:53:12 oleg133-client kernel: Aug 15 06:53:12 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:53:12 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:53:12 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:53:12 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:53:12 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:53:12 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:53:12 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:53:12 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:53:12 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:53:12 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:53:12 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:53:12 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:53:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:53:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:53:28 oleg133-client kernel: Modules linked in: Aug 15 06:53:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:53:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:53:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:53:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:53:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:53:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:53:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:53:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:53:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:53:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:53:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:53:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:53:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:53:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:53:28 oleg133-client kernel: Call Trace: Aug 15 06:53:28 oleg133-client kernel: Aug 15 06:53:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:53:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:53:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:53:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:53:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:53:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:53:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:53:28 oleg133-client kernel: Aug 15 06:53:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:53:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:53:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:53:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:53:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:53:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:53:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:53:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:53:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:53:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:53:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:53:28 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:53:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:53:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:53:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:53:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:53:28 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:53:28 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:53:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:53:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:53:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:53:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:53:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:53:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:53:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:53:28 oleg133-client kernel: Call Trace: Aug 15 06:53:28 oleg133-client kernel: Aug 15 06:53:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:53:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:53:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:53:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:53:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:53:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:53:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:53:28 oleg133-client kernel: Aug 15 06:53:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:53:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:53:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:53:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:53:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:53:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:53:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:53:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:53:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:53:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:53:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:53:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:53:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:53:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:53:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:53:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:53:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:53:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:53:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:53:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:53:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:53:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:53:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:53:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:53:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:53:40 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:53:40 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:53:40 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:53:40 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:53:40 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:53:40 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:53:40 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:53:40 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:53:40 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:53:40 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:53:40 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:53:40 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:53:40 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:53:40 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:53:40 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:53:40 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:53:40 oleg133-client kernel: Call Trace: Aug 15 06:53:40 oleg133-client kernel: Aug 15 06:53:40 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:53:40 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:53:40 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:53:40 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:53:40 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:53:40 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:53:40 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:53:40 oleg133-client kernel: Aug 15 06:53:40 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:53:40 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:53:40 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:53:40 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:53:40 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:53:40 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:53:40 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:53:40 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:53:40 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:53:40 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:53:40 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:53:40 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:53:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:53:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:53:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:53:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:53:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:53:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:53:56 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:53:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:53:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:53:56 oleg133-client kernel: RDX: 000000000000679e RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:53:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:53:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:53:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:53:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:53:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:53:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:53:56 oleg133-client kernel: Call Trace: Aug 15 06:53:56 oleg133-client kernel: Aug 15 06:53:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:53:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:53:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:53:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:53:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:53:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:53:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:53:56 oleg133-client kernel: Aug 15 06:53:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:53:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:53:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:53:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:53:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:53:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:53:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:53:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:53:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:53:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:53:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:53:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:53:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:53:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:53:56 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:53:56 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:53:56 oleg133-client kernel: RAX: 0000000000004125 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:53:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:53:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:53:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:53:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:53:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:53:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:53:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:53:56 oleg133-client kernel: Call Trace: Aug 15 06:53:56 oleg133-client kernel: Aug 15 06:53:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:53:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:53:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:53:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:53:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:53:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:53:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:53:56 oleg133-client kernel: Aug 15 06:53:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:53:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:53:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:53:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:53:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:53:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:53:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:53:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:53:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:53:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:53:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:53:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:53:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:53:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:53:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:53:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:53:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:53:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:53:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:53:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:53:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:53:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:53:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:53:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:54:08 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:54:08 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:54:08 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:54:08 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:54:08 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:54:08 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:54:08 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:54:08 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:54:08 oleg133-client kernel: RAX: 0000000000007098 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:54:08 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:54:08 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:54:08 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:54:08 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:54:08 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:54:08 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:54:08 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:54:08 oleg133-client kernel: Call Trace: Aug 15 06:54:08 oleg133-client kernel: Aug 15 06:54:08 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:54:08 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:54:08 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:54:08 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:54:08 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:54:08 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:54:08 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:54:08 oleg133-client kernel: Aug 15 06:54:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:54:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:54:08 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:54:08 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:54:08 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:54:08 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:54:08 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:54:08 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:54:08 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:54:08 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:54:08 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:54:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:54:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:54:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:54:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:54:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:54:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:54:24 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:54:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:54:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:54:24 oleg133-client kernel: RDX: 0000000000007f85 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:54:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:54:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:54:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:54:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:54:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:54:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:54:24 oleg133-client kernel: Call Trace: Aug 15 06:54:24 oleg133-client kernel: Aug 15 06:54:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:54:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:54:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:54:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:54:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:54:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:54:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:54:24 oleg133-client kernel: Aug 15 06:54:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:54:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:54:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:54:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:54:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:54:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:54:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:54:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:54:24 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:54:24 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:54:24 oleg133-client kernel: RAX: 0000000000003d81 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:54:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:54:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:54:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:54:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:54:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:54:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:54:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:54:24 oleg133-client kernel: Call Trace: Aug 15 06:54:24 oleg133-client kernel: Aug 15 06:54:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:54:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:54:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:54:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:54:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:54:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:54:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:54:24 oleg133-client kernel: Aug 15 06:54:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:54:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:54:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:54:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:54:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:54:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:54:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:54:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:54:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:54:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:54:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:54:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:54:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:54:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:54:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:54:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:54:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:54:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:54:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:54:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:54:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:54:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:54:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:54:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:54:24 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:54:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:54:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:54:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:54:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:54:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:54:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:54:36 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:54:36 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:54:36 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:54:36 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:54:36 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:54:36 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:54:36 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:54:36 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:54:36 oleg133-client kernel: RAX: 000000000000788d RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:54:36 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:54:36 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:54:36 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:54:36 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:54:36 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:54:36 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:54:36 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:54:36 oleg133-client kernel: Call Trace: Aug 15 06:54:36 oleg133-client kernel: Aug 15 06:54:36 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:54:36 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:54:36 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:54:36 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:54:36 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:54:36 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:54:36 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:54:36 oleg133-client kernel: Aug 15 06:54:36 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:54:36 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:54:36 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:54:36 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:54:36 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:54:36 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:54:36 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:54:36 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:54:36 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:54:36 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:54:36 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:54:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:54:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:54:52 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:54:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs Aug 15 06:54:52 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror Aug 15 06:54:52 oleg133-client kernel: ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:54:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:54:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:54:52 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:54:52 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:54:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:54:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:54:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:54:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:54:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:54:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:54:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:54:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:54:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:54:52 oleg133-client kernel: Call Trace: Aug 15 06:54:52 oleg133-client kernel: Aug 15 06:54:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:54:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:54:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:54:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:54:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:54:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:54:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:54:52 oleg133-client kernel: Aug 15 06:54:52 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:54:52 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:54:52 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:54:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:54:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:54:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:54:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: Aug 15 06:54:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:54:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:54:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:54:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:54:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 Aug 15 06:54:52 oleg133-client kernel: RAX: 000000000000110e RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:54:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:54:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:54:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:54:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:54:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:54:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:54:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:54:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:54:52 oleg133-client kernel: Call Trace: Aug 15 06:54:52 oleg133-client kernel: Aug 15 06:54:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:54:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:54:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:54:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:54:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:54:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:54:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:54:52 oleg133-client kernel: Aug 15 06:54:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:54:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:54:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:54:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:54:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:54:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:54:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:54:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:54:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:54:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:54:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:54:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:54:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:54:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:54:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:54:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:54:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:54:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:54:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:54:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:54:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:54:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:54:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:54:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:54:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:54:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:54:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:54:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:54:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:55:04 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:55:04 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:55:04 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:55:04 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:55:04 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:55:04 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:55:04 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:55:04 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:55:04 oleg133-client kernel: RAX: 0000000000005786 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:55:04 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:55:04 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:55:04 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:55:04 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:55:04 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:55:04 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:55:04 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:55:04 oleg133-client kernel: Call Trace: Aug 15 06:55:04 oleg133-client kernel: Aug 15 06:55:04 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:55:04 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:55:04 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:55:04 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:55:04 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:55:04 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:55:04 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:55:04 oleg133-client kernel: Aug 15 06:55:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:55:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:55:04 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:55:04 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:55:04 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:55:04 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:55:04 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:55:04 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:55:04 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:55:04 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:55:04 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:55:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:55:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:55:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:55:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:55:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:55:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:55:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:55:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:55:20 oleg133-client kernel: RAX: 0000000000005a11 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:55:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:55:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:55:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:55:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:55:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:55:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:55:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:55:20 oleg133-client kernel: Call Trace: Aug 15 06:55:20 oleg133-client kernel: Aug 15 06:55:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:55:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:55:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:55:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:55:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:55:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:55:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:55:20 oleg133-client kernel: Aug 15 06:55:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:55:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:55:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:55:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:55:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:55:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:55:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:55:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:55:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:55:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:55:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:55:20 oleg133-client kernel: RDX: 0000000000004a9f RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:55:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:55:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:55:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:55:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:55:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:55:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:55:20 oleg133-client kernel: Call Trace: Aug 15 06:55:20 oleg133-client kernel: Aug 15 06:55:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:55:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:55:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:55:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:55:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:55:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:55:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:55:20 oleg133-client kernel: Aug 15 06:55:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:55:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:55:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:55:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:55:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:55:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:55:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:55:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:55:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:55:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:55:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:55:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:55:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:55:21 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:55:21 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:55:21 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:55:21 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:55:21 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:55:21 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:55:21 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:55:21 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:55:21 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:55:21 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:55:21 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:55:21 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:55:21 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:55:21 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:55:21 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:55:21 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:55:21 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:55:32 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:55:32 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:55:32 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:55:32 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:55:32 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:55:32 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:55:32 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:55:32 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:55:32 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:55:32 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:55:32 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:55:32 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:55:32 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:55:32 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:55:32 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:55:32 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:55:32 oleg133-client kernel: Call Trace: Aug 15 06:55:32 oleg133-client kernel: Aug 15 06:55:32 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:55:32 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:55:32 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:55:32 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:55:32 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:55:32 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:55:32 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:55:32 oleg133-client kernel: Aug 15 06:55:32 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:55:32 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:55:32 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:55:32 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:55:32 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:55:32 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:55:32 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:55:32 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:55:32 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:55:32 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:55:32 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:55:32 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:55:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:55:34 oleg133-client kernel: rcu: #0111-....: (1108348 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=320835 Aug 15 06:55:34 oleg133-client kernel: #011(t=1680041 jiffies g=278549 q=9657) Aug 15 06:55:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:55:34 oleg133-client kernel: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 Aug 15 06:55:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:55:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:55:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:55:34 oleg133-client kernel: Call Trace: Aug 15 06:55:34 oleg133-client kernel: Aug 15 06:55:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:55:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:55:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:55:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:55:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:55:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:55:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:55:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:55:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:55:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:55:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:55:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:55:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:55:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:55:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:55:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:55:34 oleg133-client kernel: Aug 15 06:55:34 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:55:34 oleg133-client kernel: ? kvm_wait+0xcd/0xe0 Aug 15 06:55:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:55:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:55:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:55:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:55:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:55:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:55:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:55:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:55:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:55:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:55:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:55:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:55:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:55:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:55:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:55:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:55:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:56:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:56:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:56:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:56:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:56:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:56:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:56:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:56:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:56:00 oleg133-client kernel: RAX: 0000000000002684 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:56:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:56:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:56:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:56:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:56:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:56:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:56:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:56:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:56:00 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:56:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:56:00 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:56:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:56:00 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:56:00 oleg133-client kernel: RAX: 000000000000202f RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:56:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:56:00 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:56:00 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:56:00 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:56:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:56:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:56:00 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:56:00 oleg133-client kernel: Call Trace: Aug 15 06:56:00 oleg133-client kernel: Aug 15 06:56:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:56:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:56:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:56:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:56:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:56:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:56:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:56:00 oleg133-client kernel: Aug 15 06:56:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:56:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:56:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:56:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:56:00 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:56:00 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:56:00 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:56:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:56:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:56:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:56:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 Aug 15 06:56:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:56:00 oleg133-client kernel: Call Trace: Aug 15 06:56:00 oleg133-client kernel: Aug 15 06:56:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:56:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:56:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:56:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:56:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:56:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:56:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:56:00 oleg133-client kernel: Aug 15 06:56:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:56:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:56:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:56:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:56:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:56:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: auth_rpcgss Aug 15 06:56:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:56:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: nfsv4 Aug 15 06:56:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:56:00 oleg133-client kernel: dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:56:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:56:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:56:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:56:00 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:56:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:56:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:56:00 oleg133-client kernel: RDX: 0000000000005193 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:56:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:56:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:56:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:56:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:56:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:56:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:56:00 oleg133-client kernel: Call Trace: Aug 15 06:56:00 oleg133-client kernel: Aug 15 06:56:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:56:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:56:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:56:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:56:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:56:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:56:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:56:00 oleg133-client kernel: Aug 15 06:56:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:56:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:56:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:56:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:56:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:56:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:56:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:56:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:56:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:56:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:56:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:56:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:56:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:56:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:56:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:56:01 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:56:01 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:56:01 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:56:01 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:56:01 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:56:01 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:56:01 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:56:01 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:56:01 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:56:01 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:56:01 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:56:01 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:56:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:56:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:56:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:56:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:56:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:56:28 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:56:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:56:28 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:56:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:56:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:56:28 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 Aug 15 06:56:28 oleg133-client kernel: RAX: 0000000000007e99 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:56:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:56:28 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:56:28 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:56:28 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:56:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:56:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:56:28 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:56:28 oleg133-client kernel: Call Trace: Aug 15 06:56:28 oleg133-client kernel: Aug 15 06:56:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:56:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:56:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:56:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:56:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:56:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:56:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:56:28 oleg133-client kernel: Aug 15 06:56:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:56:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:56:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:56:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:56:28 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:56:28 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:56:28 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:56:28 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:56:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:56:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:56:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:56:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:56:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:56:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:56:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:56:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:56:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:56:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:56:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:56:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:56:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:56:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:56:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:56:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:56:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:56:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:56:28 oleg133-client kernel: Call Trace: Aug 15 06:56:28 oleg133-client kernel: Aug 15 06:56:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:56:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:56:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:56:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:56:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:56:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:56:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:56:28 oleg133-client kernel: Aug 15 06:56:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:56:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:56:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:56:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:56:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:56:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:56:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:56:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:56:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:56:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:56:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:56:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:56:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:56:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:56:28 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 Aug 15 06:56:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:56:28 oleg133-client kernel: ORIG_RAX: ffffffffffffff13 Aug 15 06:56:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:56:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:56:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:56:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:56:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:56:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:56:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:56:29 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:56:29 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:56:29 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:56:29 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:56:29 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:56:29 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:56:29 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:56:29 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:56:29 oleg133-client kernel: Call Trace: Aug 15 06:56:29 oleg133-client kernel: Aug 15 06:56:29 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:56:29 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:56:29 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:56:29 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:56:29 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:56:29 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:56:29 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:56:29 oleg133-client kernel: Aug 15 06:56:29 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:56:29 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:56:29 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:56:29 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:56:29 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:56:29 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:56:29 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:56:29 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:56:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:56:29 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:56:29 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:56:29 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:56:29 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:56:29 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:56:29 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:56:29 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:56:29 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:56:29 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:56:29 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:56:29 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:56:29 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:56:29 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:56:29 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:56:29 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:56:29 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:56:29 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:56:29 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:56:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:56:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:56:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:56:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:56:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:56:56 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:56:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:56:56 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:56:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:56:56 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:56:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:56:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:56:56 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:56:56 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:56:56 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:56:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:56:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:56:56 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:56:56 oleg133-client kernel: Call Trace: Aug 15 06:56:56 oleg133-client kernel: Aug 15 06:56:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:56:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:56:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:56:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:56:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:56:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:56:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:56:56 oleg133-client kernel: Aug 15 06:56:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:56:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:56:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:56:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:56:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:56:56 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:56:56 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:56:56 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:56:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:56:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:56:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:56:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:56:56 oleg133-client kernel: Modules linked in: Aug 15 06:56:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:56:56 oleg133-client kernel: lustre(O) Aug 15 06:56:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:56:56 oleg133-client kernel: osp(O) Aug 15 06:56:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:56:56 oleg133-client kernel: ofd(O) Aug 15 06:56:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 Aug 15 06:56:56 oleg133-client kernel: lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:56:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:56:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:56:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:56:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:56:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:56:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:56:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:56:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:56:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:56:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:56:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:56:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:56:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:56:56 oleg133-client kernel: Call Trace: Aug 15 06:56:56 oleg133-client kernel: Aug 15 06:56:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:56:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:56:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:56:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:56:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:56:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:56:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:56:56 oleg133-client kernel: Aug 15 06:56:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:56:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:56:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:56:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:56:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:56:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:56:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: ORIG_RAX: ffffffffffffff13 Aug 15 06:56:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:56:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:56:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:56:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:56:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:56:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:56:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:56:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:56:56 oleg133-client kernel: Call Trace: Aug 15 06:56:56 oleg133-client kernel: Aug 15 06:56:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:56:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:56:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:56:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:56:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:56:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:56:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:56:56 oleg133-client kernel: Aug 15 06:56:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:56:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:56:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:56:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:56:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:56:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:56:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:56:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:56:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:56:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:56:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:56:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:56:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:56:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:56:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:56:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:56:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:56:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:56:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:56:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:56:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:56:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:56:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:56:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:56:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:56:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:56:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:56:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:56:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:56:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:57:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:57:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:57:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:57:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod Aug 15 06:57:24 oleg133-client kernel: Modules linked in: Aug 15 06:57:24 oleg133-client kernel: sha512_ssse3 Aug 15 06:57:24 oleg133-client kernel: lustre(O) Aug 15 06:57:24 oleg133-client kernel: sha512_generic Aug 15 06:57:24 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) Aug 15 06:57:24 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:57:24 oleg133-client kernel: mgs(O) lquota(O) Aug 15 06:57:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:57:24 oleg133-client kernel: lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:57:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:57:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:57:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:57:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:57:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:57:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:57:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:57:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:57:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:57:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:57:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:57:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:57:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:57:24 oleg133-client kernel: Call Trace: Aug 15 06:57:24 oleg133-client kernel: Aug 15 06:57:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:57:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:57:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:57:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:57:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:57:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:57:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:57:24 oleg133-client kernel: Aug 15 06:57:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:57:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:57:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:57:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:57:24 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:57:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: Aug 15 06:57:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:57:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:57:24 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:57:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:57:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:57:24 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:57:24 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:57:24 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:57:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:57:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:57:24 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:57:24 oleg133-client kernel: Call Trace: Aug 15 06:57:24 oleg133-client kernel: Aug 15 06:57:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:57:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:57:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:57:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:57:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:57:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:57:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:57:24 oleg133-client kernel: Aug 15 06:57:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:57:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:57:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:57:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:57:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:57:24 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:57:24 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:57:24 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:57:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:57:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:57:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:57:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:57:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:57:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:57:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:57:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:57:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:57:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:57:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:57:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:57:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:57:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:57:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:57:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:57:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:57:24 oleg133-client kernel: Call Trace: Aug 15 06:57:24 oleg133-client kernel: Aug 15 06:57:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:57:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:57:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:57:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:57:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:57:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:57:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:57:24 oleg133-client kernel: Aug 15 06:57:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:57:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:57:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:57:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:57:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:57:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:57:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:57:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:57:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:57:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:57:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:57:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:57:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:57:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:57:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:57:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:57:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:57:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:57:25 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:57:25 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:57:25 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:57:25 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:57:25 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:57:25 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:57:25 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:57:25 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:57:25 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:57:25 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:57:25 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:57:25 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:57:25 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:57:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:57:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:57:52 oleg133-client kernel: Modules linked in: Aug 15 06:57:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:57:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:57:52 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:57:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:57:52 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:57:52 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:57:52 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:57:52 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:57:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:57:52 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:57:52 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:57:52 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:57:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:57:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:57:52 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:57:52 oleg133-client kernel: Call Trace: Aug 15 06:57:52 oleg133-client kernel: Aug 15 06:57:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:57:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:57:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:57:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:57:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:57:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:57:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:57:52 oleg133-client kernel: Aug 15 06:57:52 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:57:52 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:57:52 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:57:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:57:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:57:52 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:57:52 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:57:52 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:57:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:57:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:57:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:57:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:57:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:57:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:57:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:57:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:57:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:57:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:57:52 oleg133-client kernel: RDX: 0000000000002d9c RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:57:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:57:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:57:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:57:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:57:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:57:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:57:52 oleg133-client kernel: Call Trace: Aug 15 06:57:52 oleg133-client kernel: Aug 15 06:57:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:57:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:57:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:57:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:57:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:57:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:57:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:57:52 oleg133-client kernel: Aug 15 06:57:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:57:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:57:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:57:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:57:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:57:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: lustre(O) osp(O) Aug 15 06:57:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:57:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:57:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:57:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:57:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:57:52 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:57:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:57:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:57:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:57:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:57:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:57:52 oleg133-client kernel: RAX: 0000000000001694 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:57:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:57:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:57:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:57:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:57:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:57:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:57:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:57:52 oleg133-client kernel: Call Trace: Aug 15 06:57:52 oleg133-client kernel: Aug 15 06:57:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:57:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:57:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:57:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:57:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:57:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:57:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:57:52 oleg133-client kernel: Aug 15 06:57:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:57:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:57:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:57:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:57:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:57:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:57:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:57:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:57:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:57:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:57:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:57:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:57:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:57:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:57:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:57:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:57:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:57:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:57:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:57:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:57:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:57:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:57:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:57:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:58:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 06:58:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 06:58:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:58:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:58:20 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:58:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:58:20 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:58:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:58:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:58:20 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:58:20 oleg133-client kernel: RAX: 000000000000691d RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:58:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:58:20 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:58:20 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:58:20 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:58:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:58:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:58:20 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:58:20 oleg133-client kernel: Call Trace: Aug 15 06:58:20 oleg133-client kernel: Aug 15 06:58:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:58:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:58:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:58:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:58:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:58:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:58:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:58:20 oleg133-client kernel: Aug 15 06:58:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:58:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:58:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:58:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:58:20 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:58:20 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:58:20 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:58:20 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:58:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:58:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:58:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:58:20 oleg133-client kernel: Modules linked in: Aug 15 06:58:20 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 06:58:20 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:58:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:58:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:58:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:58:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:58:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:58:20 oleg133-client kernel: RAX: 000000000000590c RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:58:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:58:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:58:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:58:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:58:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:58:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:58:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:58:20 oleg133-client kernel: Call Trace: Aug 15 06:58:20 oleg133-client kernel: Aug 15 06:58:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:58:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:58:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:58:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:58:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:58:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:58:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:58:20 oleg133-client kernel: Aug 15 06:58:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:58:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:58:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:58:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:58:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:58:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:58:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:58:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:58:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:58:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:58:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:58:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:58:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:58:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:58:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:58:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:58:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:58:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:58:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:58:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:58:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:58:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:58:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:58:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:58:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:58:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:58:21 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:58:21 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:58:21 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:58:21 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:58:21 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:58:21 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:58:21 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:58:21 oleg133-client kernel: RDX: 0000000000007cea RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:58:21 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:58:21 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:58:21 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:58:21 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:58:21 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:58:21 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:58:21 oleg133-client kernel: Call Trace: Aug 15 06:58:21 oleg133-client kernel: Aug 15 06:58:21 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:58:21 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:58:21 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:58:21 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:58:21 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:58:21 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:58:21 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:58:21 oleg133-client kernel: Aug 15 06:58:21 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:58:21 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:58:21 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:58:21 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:58:21 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:58:21 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:58:21 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:58:21 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:58:21 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:58:21 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:58:21 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:58:21 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:58:21 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:58:21 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:58:21 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:58:21 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:58:21 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:58:21 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:58:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 06:58:34 oleg133-client kernel: rcu: #0111-....: (1236131 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=356529 Aug 15 06:58:34 oleg133-client kernel: #011(t=1860044 jiffies g=278549 q=10332) Aug 15 06:58:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 06:58:34 oleg133-client kernel: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 Aug 15 06:58:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 06:58:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:58:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:58:34 oleg133-client kernel: Call Trace: Aug 15 06:58:34 oleg133-client kernel: Aug 15 06:58:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 06:58:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 06:58:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 06:58:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 06:58:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 06:58:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 06:58:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 06:58:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 06:58:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 06:58:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 06:58:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 06:58:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:58:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 06:58:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:58:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:58:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:58:34 oleg133-client kernel: Aug 15 06:58:34 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:58:34 oleg133-client kernel: ? kvm_wait+0xcd/0xe0 Aug 15 06:58:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:58:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 06:58:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 06:58:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:58:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:58:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:58:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:58:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:58:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:58:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:58:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:58:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:58:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:58:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:58:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 06:58:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:58:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 06:58:48 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:58:48 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:58:48 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:58:48 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:58:48 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:58:48 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:58:48 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:58:48 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:58:48 oleg133-client kernel: RAX: 0000000000005395 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:58:48 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:58:48 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:58:48 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:58:48 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:58:48 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:58:48 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:58:48 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:58:48 oleg133-client kernel: Call Trace: Aug 15 06:58:48 oleg133-client kernel: Aug 15 06:58:48 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:58:48 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:58:48 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:58:48 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:58:48 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:58:48 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:58:48 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:58:48 oleg133-client kernel: Aug 15 06:58:48 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:58:48 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:58:48 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:58:48 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:58:48 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:58:48 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:58:48 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:58:48 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:58:48 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:58:48 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:58:48 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:59:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:59:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:59:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:59:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:59:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:59:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:59:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:59:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:59:00 oleg133-client kernel: RAX: 0000000000002115 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:59:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:59:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:59:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:59:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:59:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:59:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:59:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:59:00 oleg133-client kernel: Call Trace: Aug 15 06:59:00 oleg133-client kernel: Aug 15 06:59:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:59:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:59:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:59:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:59:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:59:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:59:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:59:00 oleg133-client kernel: Aug 15 06:59:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:59:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:59:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:59:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:59:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:59:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:59:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:59:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:59:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:59:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:59:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:59:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:59:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:59:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:59:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:59:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:59:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:59:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:59:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:59:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:59:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:59:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:59:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:59:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:59:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:59:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:59:01 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:59:01 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:59:01 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:59:01 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:59:01 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:59:01 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:59:01 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:59:01 oleg133-client kernel: RDX: 0000000000006cf1 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:59:01 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:59:01 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:59:01 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:59:01 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:59:01 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:59:01 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:59:01 oleg133-client kernel: Call Trace: Aug 15 06:59:01 oleg133-client kernel: Aug 15 06:59:01 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:59:01 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:59:01 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:59:01 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:59:01 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:59:01 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:59:01 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:59:01 oleg133-client kernel: Aug 15 06:59:01 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:59:01 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:59:01 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:59:01 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:59:01 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:59:01 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:59:01 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:59:01 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:59:01 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:59:01 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:59:01 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:59:01 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:59:01 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:59:01 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:59:01 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:59:01 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:59:01 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:59:01 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:59:16 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 06:59:16 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:59:16 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:59:16 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:59:16 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:59:16 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:59:16 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:59:16 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:59:16 oleg133-client kernel: RAX: 0000000000007d1c RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:59:16 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:59:16 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:59:16 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:59:16 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:59:16 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:59:16 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:59:16 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:59:16 oleg133-client kernel: Call Trace: Aug 15 06:59:16 oleg133-client kernel: Aug 15 06:59:16 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:59:16 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:59:16 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:59:16 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:59:16 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:59:16 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:59:16 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:59:16 oleg133-client kernel: Aug 15 06:59:16 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:59:16 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:59:16 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:59:16 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:59:16 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:59:16 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:59:16 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:59:16 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:59:16 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:59:16 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:59:16 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:59:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:59:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:59:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:59:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:59:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:59:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:59:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:59:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:59:28 oleg133-client kernel: RAX: 00000000000004e3 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:59:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:59:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:59:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:59:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:59:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:59:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:59:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:59:28 oleg133-client kernel: Call Trace: Aug 15 06:59:28 oleg133-client kernel: Aug 15 06:59:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:59:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:59:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:59:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:59:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:59:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:59:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:59:28 oleg133-client kernel: Aug 15 06:59:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:59:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:59:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:59:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:59:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:59:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:59:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:59:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:59:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:59:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:59:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:59:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:59:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:59:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:59:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:59:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:59:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:59:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:59:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:59:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:59:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:59:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:59:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:59:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:59:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:59:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:59:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:59:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 06:59:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:59:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:59:28 oleg133-client kernel: RDX: 0000000000000a0d RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:59:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:59:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:59:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:59:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:59:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:59:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:59:28 oleg133-client kernel: Call Trace: Aug 15 06:59:28 oleg133-client kernel: Aug 15 06:59:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:59:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:59:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:59:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:59:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:59:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:59:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:59:28 oleg133-client kernel: Aug 15 06:59:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 06:59:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:59:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:59:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:59:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:59:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:59:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:59:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:59:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:59:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:59:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:59:28 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 06:59:44 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 06:59:44 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:59:44 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:59:44 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:59:44 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 06:59:44 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:59:44 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 06:59:44 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 06:59:44 oleg133-client kernel: RAX: 0000000000000b87 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:59:44 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 06:59:44 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:59:44 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 06:59:44 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 06:59:44 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 06:59:44 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:59:44 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 06:59:44 oleg133-client kernel: Call Trace: Aug 15 06:59:44 oleg133-client kernel: Aug 15 06:59:44 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:59:44 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:59:44 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:59:44 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:59:44 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:59:44 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:59:44 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:59:44 oleg133-client kernel: Aug 15 06:59:44 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 06:59:44 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:59:44 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:59:44 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:59:44 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 06:59:44 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 06:59:44 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 06:59:44 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 06:59:44 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:59:44 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:59:44 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 06:59:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 06:59:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 06:59:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) Aug 15 06:59:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) Aug 15 06:59:56 oleg133-client kernel: ofd(O) lod(O) Aug 15 06:59:56 oleg133-client kernel: lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:59:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:59:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:59:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:59:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:59:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:59:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 06:59:56 oleg133-client kernel: mdt(O) Aug 15 06:59:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 06:59:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 06:59:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 06:59:56 oleg133-client kernel: mdd(O) Aug 15 06:59:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 06:59:56 oleg133-client kernel: mgs(O) Aug 15 06:59:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 06:59:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:59:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 06:59:56 oleg133-client kernel: Call Trace: Aug 15 06:59:56 oleg133-client kernel: Aug 15 06:59:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:59:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:59:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:59:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:59:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:59:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:59:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:59:56 oleg133-client kernel: Aug 15 06:59:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:59:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:59:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 06:59:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:59:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:59:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 06:59:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 06:59:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 06:59:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 06:59:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 06:59:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 06:59:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 06:59:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 06:59:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 06:59:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 06:59:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 06:59:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 06:59:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 06:59:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 06:59:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 06:59:56 oleg133-client kernel: Call Trace: Aug 15 06:59:56 oleg133-client kernel: Aug 15 06:59:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 06:59:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 06:59:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 06:59:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 06:59:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 06:59:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 06:59:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 06:59:56 oleg133-client kernel: Aug 15 06:59:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 06:59:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 06:59:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 06:59:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 06:59:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 06:59:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 06:59:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:59:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 06:59:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 06:59:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 06:59:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 06:59:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 06:59:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 06:59:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 06:59:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 06:59:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 06:59:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 06:59:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 06:59:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 06:59:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 06:59:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 06:59:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 06:59:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 06:59:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 06:59:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 06:59:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 06:59:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 06:59:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 06:59:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 06:59:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:00:12 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:00:12 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:00:12 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:00:12 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:00:12 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:00:12 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:00:12 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:00:12 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:00:12 oleg133-client kernel: RAX: 0000000000007f89 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:00:12 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:00:12 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:00:12 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:00:12 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:00:12 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:00:12 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:00:12 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:00:12 oleg133-client kernel: Call Trace: Aug 15 07:00:12 oleg133-client kernel: Aug 15 07:00:12 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:00:12 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:00:12 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:00:12 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:00:12 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:00:12 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:00:12 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:00:12 oleg133-client kernel: Aug 15 07:00:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:00:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:00:12 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:00:12 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:00:12 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:00:12 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:00:12 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:00:12 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:00:12 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:00:12 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:00:12 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:00:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:00:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:00:24 oleg133-client kernel: Modules linked in: Aug 15 07:00:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:00:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:00:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:00:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:00:24 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:00:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:00:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:00:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:00:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:00:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:00:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:00:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:00:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:00:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:00:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:00:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:00:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:00:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:00:24 oleg133-client kernel: Call Trace: Aug 15 07:00:24 oleg133-client kernel: Aug 15 07:00:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:00:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:00:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:00:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:00:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:00:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:00:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:00:24 oleg133-client kernel: Aug 15 07:00:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:00:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:00:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:00:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:00:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:00:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:00:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:00:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:00:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:00:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:00:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:00:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:00:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:00:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:00:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:00:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:00:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:00:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:00:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:00:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:00:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:00:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:00:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:00:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:00:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:00:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:00:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:00:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:00:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:00:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:00:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:00:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:00:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:00:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:00:24 oleg133-client kernel: Call Trace: Aug 15 07:00:24 oleg133-client kernel: Aug 15 07:00:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:00:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:00:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:00:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:00:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:00:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:00:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:00:24 oleg133-client kernel: Aug 15 07:00:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:00:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:00:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:00:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:00:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:00:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:00:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:00:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:00:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:00:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:00:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:00:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:00:40 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:00:40 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:00:40 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:00:40 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:00:40 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:00:40 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:00:40 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:00:40 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:00:40 oleg133-client kernel: RAX: 00000000000025ec RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:00:40 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:00:40 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:00:40 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:00:40 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:00:40 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:00:40 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:00:40 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:00:40 oleg133-client kernel: Call Trace: Aug 15 07:00:40 oleg133-client kernel: Aug 15 07:00:40 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:00:40 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:00:40 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:00:40 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:00:40 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:00:40 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:00:40 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:00:40 oleg133-client kernel: Aug 15 07:00:40 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:00:40 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:00:40 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:00:40 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:00:40 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:00:40 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:00:40 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:00:40 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:00:40 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:00:40 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:00:40 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:00:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:00:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:00:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:00:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:00:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:00:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:00:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:00:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:00:52 oleg133-client kernel: RAX: 000000000000395c RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:00:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:00:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:00:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:00:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:00:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:00:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:00:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:00:52 oleg133-client kernel: Call Trace: Aug 15 07:00:52 oleg133-client kernel: Aug 15 07:00:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:00:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:00:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:00:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:00:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:00:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:00:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:00:52 oleg133-client kernel: Aug 15 07:00:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:00:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:00:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:00:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:00:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:00:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:00:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:00:52 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 07:00:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:00:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:00:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:00:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:00:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:00:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:00:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:00:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:00:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:00:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:00:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:00:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:00:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:00:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:00:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:00:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:00:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:00:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:00:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:00:53 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:00:53 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:00:53 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:00:53 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:00:53 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:00:53 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:00:53 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:00:53 oleg133-client kernel: RDX: 0000000000007d5e RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:00:53 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:00:53 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:00:53 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:00:53 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:00:53 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:00:53 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:00:53 oleg133-client kernel: Call Trace: Aug 15 07:00:53 oleg133-client kernel: Aug 15 07:00:53 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:00:53 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:00:53 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:00:53 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:00:53 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:00:53 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:00:53 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:00:53 oleg133-client kernel: Aug 15 07:00:53 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:00:53 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:00:53 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:00:53 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:00:53 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:00:53 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:00:53 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:00:53 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:00:53 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:00:53 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:00:53 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:00:53 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:00:53 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:00:53 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:00:53 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:00:53 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:00:53 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:00:53 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:01:01 oleg133-client CROND[37000]: (root) CMD (run-parts /etc/cron.hourly) Aug 15 07:01:01 oleg133-client run-parts[37000]: (/etc/cron.hourly) starting 0anacron Aug 15 07:01:01 oleg133-client run-parts[37000]: (/etc/cron.hourly) finished 0anacron Aug 15 07:01:08 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:01:08 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:01:08 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:01:08 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:01:08 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:01:08 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:01:08 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:01:08 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:01:08 oleg133-client kernel: RAX: 0000000000006d89 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:01:08 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:01:08 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:01:08 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:01:08 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:01:08 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:01:08 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:01:08 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:01:08 oleg133-client kernel: Call Trace: Aug 15 07:01:08 oleg133-client kernel: Aug 15 07:01:08 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:01:08 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:01:08 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:01:08 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:01:08 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:01:08 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:01:08 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:01:08 oleg133-client kernel: Aug 15 07:01:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:01:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:01:08 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:01:08 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:01:08 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:01:08 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:01:08 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:01:08 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:01:08 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:01:08 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:01:08 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:01:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:01:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:01:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) Aug 15 07:01:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:01:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:01:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:01:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:01:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:01:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:01:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:01:20 oleg133-client kernel: RDX: 00000000000045da RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:01:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:01:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:01:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:01:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:01:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:01:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:01:20 oleg133-client kernel: Call Trace: Aug 15 07:01:20 oleg133-client kernel: Aug 15 07:01:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:01:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:01:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:01:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:01:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:01:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:01:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:01:20 oleg133-client kernel: Aug 15 07:01:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:01:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:01:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:01:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:01:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:01:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:01:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:01:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:01:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:01:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:01:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:01:20 oleg133-client kernel: RAX: 00000000000035a3 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:01:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:01:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:01:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:01:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:01:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:01:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:01:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:01:20 oleg133-client kernel: Call Trace: Aug 15 07:01:20 oleg133-client kernel: Aug 15 07:01:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:01:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:01:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:01:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:01:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:01:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:01:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:01:20 oleg133-client kernel: Aug 15 07:01:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:01:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:01:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:01:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:01:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:01:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:01:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:01:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:01:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:01:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:01:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:01:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:01:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:01:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:01:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:01:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:01:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:01:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:01:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:01:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:01:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:01:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:01:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:01:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:01:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:01:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:01:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:01:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:01:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:01:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 07:01:34 oleg133-client kernel: rcu: #0111-....: (1364768 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=392460 Aug 15 07:01:34 oleg133-client kernel: #011(t=2040047 jiffies g=278549 q=11529) Aug 15 07:01:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 07:01:34 oleg133-client kernel: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 Aug 15 07:01:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 07:01:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:01:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:01:34 oleg133-client kernel: Call Trace: Aug 15 07:01:34 oleg133-client kernel: Aug 15 07:01:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 07:01:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 07:01:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 07:01:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 07:01:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 07:01:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 07:01:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 07:01:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 07:01:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 07:01:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 07:01:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 07:01:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:01:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 07:01:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:01:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:01:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:01:34 oleg133-client kernel: Aug 15 07:01:34 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:01:34 oleg133-client kernel: ? kvm_wait+0xcd/0xe0 Aug 15 07:01:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:01:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 07:01:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 07:01:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:01:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:01:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:01:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:01:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:01:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:01:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:01:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:01:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:01:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:01:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:01:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 07:01:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:01:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 07:01:36 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:01:36 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:01:36 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:01:36 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:01:36 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:01:36 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:01:36 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:01:36 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:01:36 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:01:36 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:01:36 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:01:36 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:01:36 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:01:36 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:01:36 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:01:36 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:01:36 oleg133-client kernel: Call Trace: Aug 15 07:01:36 oleg133-client kernel: Aug 15 07:01:36 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:01:36 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:01:36 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:01:36 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:01:36 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:01:36 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:01:36 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:01:36 oleg133-client kernel: Aug 15 07:01:36 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:01:36 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:01:36 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:01:36 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:01:36 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:01:36 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:01:36 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:01:36 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:01:36 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:01:36 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:01:36 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:01:36 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:02:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:02:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:02:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:02:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:02:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:02:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:02:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:02:00 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 Aug 15 07:02:00 oleg133-client kernel: Modules linked in: Aug 15 07:02:00 oleg133-client kernel: ORIG_RAX: ffffffffffffff13 Aug 15 07:02:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:02:00 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:02:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:02:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:02:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:02:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:02:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:02:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:02:00 oleg133-client kernel: Call Trace: Aug 15 07:02:00 oleg133-client kernel: Aug 15 07:02:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:02:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:02:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:02:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:02:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:02:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:02:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:02:00 oleg133-client kernel: Aug 15 07:02:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:02:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:02:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:02:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:02:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:02:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:02:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:02:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:02:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:02:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:02:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:02:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:02:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:02:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:02:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:02:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:02:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:02:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:02:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:02:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:02:00 oleg133-client kernel: Call Trace: Aug 15 07:02:00 oleg133-client kernel: Aug 15 07:02:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:02:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:02:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:02:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:02:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:02:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:02:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:02:00 oleg133-client kernel: Aug 15 07:02:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:02:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:02:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:02:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:02:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:02:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:02:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:02:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:02:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:02:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:02:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:02:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:02:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:02:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:02:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:02:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:02:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:02:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:02:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:02:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:02:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:02:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:02:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:02:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:02:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:02:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:02:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:02:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:02:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:02:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:02:04 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:02:04 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:02:04 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:02:04 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:02:04 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:02:04 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:02:04 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:02:04 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:02:04 oleg133-client kernel: RAX: 0000000000007e8f RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:02:04 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:02:04 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:02:04 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:02:04 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:02:04 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:02:04 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:02:04 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:02:04 oleg133-client kernel: Call Trace: Aug 15 07:02:04 oleg133-client kernel: Aug 15 07:02:04 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:02:04 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:02:04 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:02:04 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:02:04 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:02:04 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:02:04 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:02:04 oleg133-client kernel: Aug 15 07:02:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:02:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:02:04 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:02:04 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:02:04 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:02:04 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:02:04 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:02:04 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:02:04 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:02:04 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:02:04 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:02:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:02:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:02:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:02:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:02:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:02:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:02:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:02:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:02:28 oleg133-client kernel: RAX: 0000000000007e29 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:02:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:02:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:02:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:02:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:02:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:02:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:02:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:02:28 oleg133-client kernel: Call Trace: Aug 15 07:02:28 oleg133-client kernel: Aug 15 07:02:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:02:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:02:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:02:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:02:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:02:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:02:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:02:28 oleg133-client kernel: Aug 15 07:02:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:02:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:02:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:02:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:02:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:02:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) Aug 15 07:02:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:02:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:02:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:02:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:02:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:02:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:02:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:02:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:02:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:02:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:02:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:02:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:02:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:02:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:02:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:02:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:02:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:02:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:02:28 oleg133-client kernel: mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:02:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:02:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:02:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:02:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:02:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:02:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:02:28 oleg133-client kernel: RDX: 00000000000006f6 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:02:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:02:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:02:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:02:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:02:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:02:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:02:28 oleg133-client kernel: Call Trace: Aug 15 07:02:28 oleg133-client kernel: Aug 15 07:02:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:02:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:02:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:02:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:02:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:02:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:02:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:02:28 oleg133-client kernel: Aug 15 07:02:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:02:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:02:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:02:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:02:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:02:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:02:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:02:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:02:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:02:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:02:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:02:32 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:02:32 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:02:32 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:02:32 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:02:32 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:02:32 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:02:32 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:02:32 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:02:32 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:02:32 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:02:32 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:02:32 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:02:32 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:02:32 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:02:32 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:02:32 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:02:32 oleg133-client kernel: Call Trace: Aug 15 07:02:32 oleg133-client kernel: Aug 15 07:02:32 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:02:32 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:02:32 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:02:32 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:02:32 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:02:32 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:02:32 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:02:32 oleg133-client kernel: Aug 15 07:02:32 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:02:32 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:02:32 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:02:32 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:02:32 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:02:32 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:02:32 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:02:32 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:02:32 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:02:32 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:02:32 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:02:32 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:02:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:02:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:02:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:02:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:02:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:02:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:02:56 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:02:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:02:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:02:56 oleg133-client kernel: RDX: 0000000000007cb9 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:02:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:02:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:02:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:02:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:02:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:02:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:02:56 oleg133-client kernel: Call Trace: Aug 15 07:02:56 oleg133-client kernel: Aug 15 07:02:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:02:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:02:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:02:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:02:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:02:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:02:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:02:56 oleg133-client kernel: Aug 15 07:02:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:02:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:02:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:02:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:02:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:02:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:02:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:02:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:02:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:02:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:02:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:02:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:02:56 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:02:56 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 Aug 15 07:02:56 oleg133-client kernel: RAX: 000000000000173a RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:02:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:02:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:02:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:02:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:02:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:02:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:02:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:02:56 oleg133-client kernel: Call Trace: Aug 15 07:02:56 oleg133-client kernel: Aug 15 07:02:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:02:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:02:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:02:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:02:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:02:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:02:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:02:56 oleg133-client kernel: Aug 15 07:02:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:02:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:02:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:02:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:02:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:02:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:02:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:02:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:02:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:02:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:02:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:02:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:02:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:02:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:02:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:02:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:02:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:02:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:02:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:02:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:02:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:02:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:02:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:02:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:02:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:02:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:02:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:02:57 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:02:57 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:02:57 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:02:57 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:02:57 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:02:57 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:02:57 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:02:57 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:02:57 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:03:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:03:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:03:00 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:03:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:03:00 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:03:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:03:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:03:00 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:03:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:03:00 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:03:00 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:03:00 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:03:00 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:03:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:03:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:03:00 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:03:00 oleg133-client kernel: Call Trace: Aug 15 07:03:00 oleg133-client kernel: Aug 15 07:03:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:03:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:03:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:03:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:03:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:03:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:03:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:03:00 oleg133-client kernel: Aug 15 07:03:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:03:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:03:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:03:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:03:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:03:00 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:03:00 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:03:00 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:03:00 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:03:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:03:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:03:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:03:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:03:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:03:24 oleg133-client kernel: Modules linked in: Aug 15 07:03:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:03:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:03:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:03:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:03:24 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:03:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:03:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:03:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:03:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:03:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:03:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:03:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:03:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:03:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:03:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:03:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:03:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:03:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:03:24 oleg133-client kernel: Call Trace: Aug 15 07:03:24 oleg133-client kernel: Aug 15 07:03:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:03:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:03:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:03:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:03:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:03:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:03:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:03:24 oleg133-client kernel: Aug 15 07:03:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:03:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:03:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:03:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:03:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:03:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:03:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:03:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:03:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:03:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:03:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:03:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:03:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:03:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:03:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:03:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:03:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:03:24 oleg133-client kernel: Call Trace: Aug 15 07:03:24 oleg133-client kernel: Aug 15 07:03:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:03:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:03:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:03:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:03:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:03:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:03:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:03:24 oleg133-client kernel: Aug 15 07:03:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:03:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:03:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:03:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:03:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:03:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:03:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:03:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:03:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:03:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:03:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:03:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:03:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:03:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:03:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:03:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:03:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:03:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:03:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:03:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:03:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:03:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:03:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:03:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:03:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:03:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:03:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:03:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:03:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:03:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:03:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:03:28 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:03:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:03:28 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:03:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:03:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:03:28 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:03:28 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:03:28 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:03:28 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:03:28 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:03:28 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:03:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:03:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:03:28 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:03:28 oleg133-client kernel: Call Trace: Aug 15 07:03:28 oleg133-client kernel: Aug 15 07:03:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:03:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:03:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:03:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:03:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:03:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:03:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:03:28 oleg133-client kernel: Aug 15 07:03:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:03:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:03:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:03:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:03:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:03:28 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:03:28 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:03:28 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:03:28 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:03:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:03:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:03:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:03:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:03:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:03:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:03:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:03:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:03:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:03:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:03:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:03:52 oleg133-client kernel: RAX: 00000000000078de RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:03:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:03:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:03:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:03:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:03:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:03:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:03:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:03:52 oleg133-client kernel: Call Trace: Aug 15 07:03:52 oleg133-client kernel: Aug 15 07:03:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:03:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:03:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:03:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:03:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:03:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:03:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:03:52 oleg133-client kernel: Aug 15 07:03:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:03:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:03:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:03:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:03:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:03:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:03:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:03:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:03:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:03:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:03:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:03:52 oleg133-client kernel: RDX: 0000000000006f21 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:03:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:03:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:03:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:03:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:03:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:03:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:03:52 oleg133-client kernel: Call Trace: Aug 15 07:03:52 oleg133-client kernel: Aug 15 07:03:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:03:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:03:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:03:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:03:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:03:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:03:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:03:52 oleg133-client kernel: Aug 15 07:03:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:03:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:03:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:03:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:03:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:03:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:03:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:03:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:03:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:03:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:03:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:03:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:03:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:03:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:03:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:03:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:03:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:03:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:03:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:03:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:03:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:03:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:03:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:03:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:03:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:03:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:03:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:03:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:03:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:03:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:03:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:03:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:03:56 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:03:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:03:56 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:03:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:03:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:03:56 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:03:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:03:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:03:56 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:03:56 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:03:56 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:03:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:03:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:03:56 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:03:56 oleg133-client kernel: Call Trace: Aug 15 07:03:56 oleg133-client kernel: Aug 15 07:03:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:03:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:03:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:03:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:03:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:03:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:03:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:03:56 oleg133-client kernel: Aug 15 07:03:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:03:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:03:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:03:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:03:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:03:56 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:03:56 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:03:56 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:03:56 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:03:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:03:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:03:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:04:01 oleg133-client anacron[12030]: Job `cron.weekly' started Aug 15 07:04:01 oleg133-client anacron[12030]: Job `cron.weekly' terminated Aug 15 07:04:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:04:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:04:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) Aug 15 07:04:20 oleg133-client kernel: Modules linked in: Aug 15 07:04:20 oleg133-client kernel: ptlrpc(O) obdclass(O) ksocklnd(O) Aug 15 07:04:20 oleg133-client kernel: lustre(O) Aug 15 07:04:20 oleg133-client kernel: lnet(O) Aug 15 07:04:20 oleg133-client kernel: osp(O) Aug 15 07:04:20 oleg133-client kernel: libcfs(O) Aug 15 07:04:20 oleg133-client kernel: ofd(O) Aug 15 07:04:20 oleg133-client kernel: rpcsec_gss_krb5 Aug 15 07:04:20 oleg133-client kernel: lod(O) Aug 15 07:04:20 oleg133-client kernel: auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 Aug 15 07:04:20 oleg133-client kernel: mdt(O) Aug 15 07:04:20 oleg133-client kernel: sha512_generic Aug 15 07:04:20 oleg133-client kernel: mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common Aug 15 07:04:20 oleg133-client kernel: Aug 15 07:04:20 oleg133-client kernel: sb_edac Aug 15 07:04:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:04:20 oleg133-client kernel: rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:04:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:04:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:04:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:04:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:04:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:04:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:04:20 oleg133-client kernel: RDX: 00000000000051e3 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:04:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:04:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:04:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:04:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:04:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:04:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:04:20 oleg133-client kernel: Call Trace: Aug 15 07:04:20 oleg133-client kernel: Aug 15 07:04:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:04:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:04:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:04:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:04:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:04:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:04:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:04:20 oleg133-client kernel: Aug 15 07:04:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:04:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:04:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:04:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:04:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:04:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:04:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:04:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:04:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:04:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:04:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:04:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:04:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:04:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:04:20 oleg133-client kernel: RAX: 0000000000000b16 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:04:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:04:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:04:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:04:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:04:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:04:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:04:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:04:20 oleg133-client kernel: Call Trace: Aug 15 07:04:20 oleg133-client kernel: Aug 15 07:04:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:04:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:04:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:04:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:04:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:04:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:04:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:04:20 oleg133-client kernel: Aug 15 07:04:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:04:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:04:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:04:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:04:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:04:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:04:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:04:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:04:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:04:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:04:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:04:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:04:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:04:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:04:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:04:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:04:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:04:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:04:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:04:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:04:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:04:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:04:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:04:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:04:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:04:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:04:24 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:04:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:04:24 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:04:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:04:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:04:24 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:04:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:04:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:04:24 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:04:24 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:04:24 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:04:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:04:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:04:24 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:04:24 oleg133-client kernel: Call Trace: Aug 15 07:04:24 oleg133-client kernel: Aug 15 07:04:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:04:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:04:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:04:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:04:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:04:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:04:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:04:24 oleg133-client kernel: Aug 15 07:04:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:04:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:04:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:04:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:04:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:04:24 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:04:24 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:04:24 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:04:24 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:04:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:04:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:04:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:04:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 07:04:34 oleg133-client kernel: rcu: #0111-....: (1508053 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=429926 Aug 15 07:04:34 oleg133-client kernel: #011(t=2220050 jiffies g=278549 q=12286) Aug 15 07:04:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 07:04:34 oleg133-client kernel: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 Aug 15 07:04:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 07:04:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:04:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:04:34 oleg133-client kernel: Call Trace: Aug 15 07:04:34 oleg133-client kernel: Aug 15 07:04:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 07:04:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 07:04:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 07:04:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 07:04:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 07:04:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 07:04:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 07:04:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 07:04:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 07:04:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 07:04:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 07:04:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:04:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 07:04:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:04:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:04:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:04:34 oleg133-client kernel: Aug 15 07:04:34 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:04:34 oleg133-client kernel: ? kvm_wait+0xcd/0xe0 Aug 15 07:04:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:04:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 07:04:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 07:04:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:04:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:04:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:04:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:04:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:04:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:04:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:04:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:04:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:04:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:04:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:04:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 07:04:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:04:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 07:04:34 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 07:04:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:04:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:04:52 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:04:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:04:52 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:04:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:04:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:04:52 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:04:52 oleg133-client kernel: RAX: 0000000000007bbd RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:04:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:04:52 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:04:52 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:04:52 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:04:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:04:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:04:52 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:04:52 oleg133-client kernel: Call Trace: Aug 15 07:04:52 oleg133-client kernel: Aug 15 07:04:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:04:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:04:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:04:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:04:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:04:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:04:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:04:52 oleg133-client kernel: Aug 15 07:04:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:04:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:04:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:04:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:04:52 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:04:52 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:04:52 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:04:52 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:04:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:04:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:04:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:05:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:05:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:05:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) Aug 15 07:05:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:05:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:05:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:05:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:05:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:05:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:05:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:05:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:05:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:05:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:05:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:05:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:05:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:05:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:05:00 oleg133-client kernel: Call Trace: Aug 15 07:05:00 oleg133-client kernel: Aug 15 07:05:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:05:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:05:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:05:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:05:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:05:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:05:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:05:00 oleg133-client kernel: Aug 15 07:05:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:05:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:05:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:05:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:05:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:05:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:05:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:05:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:05:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:05:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:05:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:05:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:05:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:05:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:05:00 oleg133-client kernel: lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:05:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:05:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:05:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:05:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:05:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:05:00 oleg133-client kernel: RAX: 0000000000007f91 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:05:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:05:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:05:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:05:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:05:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:05:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:05:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:05:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:05:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:05:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:05:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:05:01 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:05:01 oleg133-client kernel: Call Trace: Aug 15 07:05:01 oleg133-client kernel: Aug 15 07:05:01 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:05:01 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:05:01 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:05:01 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:05:01 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:05:01 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:05:01 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:05:01 oleg133-client kernel: Aug 15 07:05:01 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:05:01 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:05:01 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:05:01 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:05:01 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:05:01 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:05:01 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:05:01 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:05:01 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:05:01 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:05:01 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:05:01 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:05:01 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:05:01 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:05:01 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:05:01 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:05:01 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:05:01 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:05:01 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:05:01 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:05:01 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:05:01 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:05:01 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:05:01 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:05:01 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:05:01 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:05:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:05:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:05:20 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:05:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:05:20 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:05:20 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:05:20 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:05:20 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:05:20 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:05:20 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:05:20 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:05:20 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:05:20 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:05:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:05:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:05:20 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:05:20 oleg133-client kernel: Call Trace: Aug 15 07:05:20 oleg133-client kernel: Aug 15 07:05:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:05:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:05:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:05:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:05:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:05:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:05:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:05:20 oleg133-client kernel: Aug 15 07:05:20 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:05:20 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:05:20 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:05:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:05:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:05:20 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:05:20 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:05:20 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:05:20 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:05:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:05:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:05:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:05:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:05:28 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 07:05:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:05:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:05:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:05:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:05:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:05:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:05:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:05:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:05:28 oleg133-client kernel: RDX: 0000000000001b4a RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:05:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:05:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:05:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:05:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:05:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:05:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:05:28 oleg133-client kernel: Call Trace: Aug 15 07:05:28 oleg133-client kernel: Aug 15 07:05:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:05:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:05:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:05:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:05:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:05:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:05:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:05:28 oleg133-client kernel: Aug 15 07:05:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:05:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:05:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:05:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:05:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:05:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:05:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:05:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:05:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:05:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:05:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:05:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:05:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:05:28 oleg133-client kernel: RAX: 000000000000348f RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:05:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:05:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:05:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:05:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:05:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:05:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:05:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:05:28 oleg133-client kernel: Call Trace: Aug 15 07:05:28 oleg133-client kernel: Aug 15 07:05:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:05:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:05:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:05:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:05:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:05:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:05:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:05:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:05:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:05:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:05:28 oleg133-client kernel: Aug 15 07:05:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:05:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:05:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:05:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:05:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:05:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:05:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:05:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:05:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:05:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:05:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:05:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:05:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:05:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:05:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:05:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:05:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:05:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:05:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:05:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:05:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:05:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:05:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:05:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:05:48 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:05:48 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:05:48 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:05:48 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:05:48 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:05:48 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:05:48 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:05:48 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:05:48 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:05:48 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:05:48 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:05:48 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:05:48 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:05:48 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:05:48 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:05:48 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:05:48 oleg133-client kernel: Call Trace: Aug 15 07:05:48 oleg133-client kernel: Aug 15 07:05:48 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:05:48 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:05:48 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:05:48 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:05:48 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:05:48 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:05:48 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:05:48 oleg133-client kernel: Aug 15 07:05:48 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:05:48 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:05:48 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:05:48 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:05:48 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:05:48 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:05:48 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:05:48 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:05:48 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:05:48 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:05:48 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:05:48 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:05:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:05:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:05:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:05:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:05:56 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 07:05:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:05:56 oleg133-client kernel: osp(O) ofd(O) lod(O) Aug 15 07:05:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:05:56 oleg133-client kernel: mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:05:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:05:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:05:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:05:56 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:05:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:05:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:05:56 oleg133-client kernel: RDX: 0000000000000237 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:05:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:05:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:05:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:05:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:05:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:05:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:05:56 oleg133-client kernel: Call Trace: Aug 15 07:05:56 oleg133-client kernel: Aug 15 07:05:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:05:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:05:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:05:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:05:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:05:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:05:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:05:56 oleg133-client kernel: Aug 15 07:05:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:05:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:05:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:05:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:05:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:05:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:05:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:05:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:05:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:05:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:05:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:05:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:05:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:05:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:05:56 oleg133-client kernel: Call Trace: Aug 15 07:05:56 oleg133-client kernel: Aug 15 07:05:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:05:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:05:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:05:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:05:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:05:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:05:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:05:56 oleg133-client kernel: Aug 15 07:05:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:05:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:05:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:05:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:05:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:05:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:05:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:05:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:05:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:05:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:05:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:05:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:05:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:05:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:05:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:05:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:05:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:05:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:05:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:05:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:05:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:05:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:05:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:05:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:05:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:05:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:05:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:05:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:05:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:05:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:05:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:06:16 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:06:16 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:06:16 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:06:16 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:06:16 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:06:16 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:06:16 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:06:16 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:06:16 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:06:16 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:06:16 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:06:16 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:06:16 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:06:16 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:06:16 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:06:16 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:06:16 oleg133-client kernel: Call Trace: Aug 15 07:06:16 oleg133-client kernel: Aug 15 07:06:16 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:06:16 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:06:16 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:06:16 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:06:16 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:06:16 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:06:16 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:06:16 oleg133-client kernel: Aug 15 07:06:16 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:06:16 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:06:16 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:06:16 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:06:16 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:06:16 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:06:16 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:06:16 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:06:16 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:06:16 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:06:16 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:06:16 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:06:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:06:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:06:24 oleg133-client kernel: Modules linked in: Aug 15 07:06:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 Aug 15 07:06:24 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:06:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:06:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:06:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:06:24 oleg133-client kernel: squashfs Aug 15 07:06:24 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:06:24 oleg133-client kernel: ata_generic Aug 15 07:06:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:06:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:06:24 oleg133-client kernel: RDX: 0000000000002106 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:06:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:06:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:06:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:06:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:06:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:06:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:06:24 oleg133-client kernel: Call Trace: Aug 15 07:06:24 oleg133-client kernel: Aug 15 07:06:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:06:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:06:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:06:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:06:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:06:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:06:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:06:24 oleg133-client kernel: Aug 15 07:06:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:06:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:06:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:06:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:06:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: crct10dif_pclmul Aug 15 07:06:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:06:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:06:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:06:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: crc32_pclmul Aug 15 07:06:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:06:24 oleg133-client kernel: crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:06:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:06:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:06:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:06:24 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:06:24 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:06:24 oleg133-client kernel: RAX: 000000000000538d RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:06:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:06:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:06:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:06:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:06:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:06:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:06:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:06:24 oleg133-client kernel: Call Trace: Aug 15 07:06:24 oleg133-client kernel: Aug 15 07:06:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:06:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:06:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:06:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:06:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:06:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:06:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:06:24 oleg133-client kernel: Aug 15 07:06:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:06:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:06:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:06:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:06:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:06:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:06:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:06:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:06:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:06:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:06:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:06:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:06:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:06:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:06:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:06:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:06:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:06:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:06:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:06:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:06:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:06:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:06:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:06:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:06:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:06:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:06:44 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:06:44 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:06:44 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:06:44 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:06:44 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:06:44 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:06:44 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:06:44 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:06:44 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:06:44 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:06:44 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:06:44 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:06:44 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:06:44 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:06:44 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:06:44 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:06:44 oleg133-client kernel: Call Trace: Aug 15 07:06:44 oleg133-client kernel: Aug 15 07:06:44 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:06:44 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:06:44 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:06:44 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:06:44 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:06:44 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:06:44 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:06:44 oleg133-client kernel: Aug 15 07:06:44 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:06:44 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:06:44 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:06:44 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:06:44 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:06:44 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:06:44 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:06:44 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:06:44 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:06:44 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:06:44 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:06:44 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:06:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:06:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:06:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:06:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:06:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:06:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:06:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:06:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:06:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:06:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:06:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:06:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:06:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:06:52 oleg133-client kernel: RDX: 0000000000007f81 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:06:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:06:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:06:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:06:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:06:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:06:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:06:52 oleg133-client kernel: Call Trace: Aug 15 07:06:52 oleg133-client kernel: Aug 15 07:06:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:06:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:06:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:06:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:06:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:06:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:06:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:06:52 oleg133-client kernel: Aug 15 07:06:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:06:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:06:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:06:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:06:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:06:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:06:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:06:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:06:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:06:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:06:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:06:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:06:52 oleg133-client kernel: RAX: 00000000000078b5 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:06:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:06:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:06:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:06:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:06:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:06:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:06:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:06:52 oleg133-client kernel: Call Trace: Aug 15 07:06:52 oleg133-client kernel: Aug 15 07:06:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:06:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:06:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:06:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:06:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:06:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:06:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:06:52 oleg133-client kernel: Aug 15 07:06:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:06:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:06:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:06:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:06:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:06:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:06:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:06:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:06:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:06:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:06:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:06:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:06:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:06:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:06:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:06:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:06:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:06:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:06:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:06:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:06:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:06:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:06:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:06:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:07:12 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:07:12 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:07:12 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:07:12 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:07:12 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:07:12 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:07:12 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:07:12 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:07:12 oleg133-client kernel: RAX: 0000000000006fec RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:07:12 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:07:12 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:07:12 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:07:12 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:07:12 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:07:12 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:07:12 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:07:12 oleg133-client kernel: Call Trace: Aug 15 07:07:12 oleg133-client kernel: Aug 15 07:07:12 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:07:12 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:07:12 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:07:12 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:07:12 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:07:12 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:07:12 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:07:12 oleg133-client kernel: Aug 15 07:07:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:07:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:07:12 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:07:12 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:07:12 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:07:12 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:07:12 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:07:12 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:07:12 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:07:12 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:07:12 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:07:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:07:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:07:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:07:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:07:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:07:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:07:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:07:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:07:20 oleg133-client kernel: RAX: 0000000000004539 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:07:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:07:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:07:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:07:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:07:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:07:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:07:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:07:20 oleg133-client kernel: Call Trace: Aug 15 07:07:20 oleg133-client kernel: Aug 15 07:07:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:07:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:07:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:07:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:07:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:07:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:07:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:07:20 oleg133-client kernel: Aug 15 07:07:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:07:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:07:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:07:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:07:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:07:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:07:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:07:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:07:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:07:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:07:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:07:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:07:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:07:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:07:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:07:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:07:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:07:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:07:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:07:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:07:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:07:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:07:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:07:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:07:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:07:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:07:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:07:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:07:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:07:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:07:20 oleg133-client kernel: RDX: 0000000000007e45 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:07:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:07:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:07:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:07:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:07:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:07:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:07:20 oleg133-client kernel: Call Trace: Aug 15 07:07:20 oleg133-client kernel: Aug 15 07:07:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:07:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:07:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:07:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:07:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:07:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:07:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:07:20 oleg133-client kernel: Aug 15 07:07:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:07:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:07:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:07:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:07:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:07:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:07:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:07:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:07:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:07:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:07:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:07:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 07:07:34 oleg133-client kernel: rcu: #0111-....: (1634355 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=465735 Aug 15 07:07:34 oleg133-client kernel: #011(t=2400054 jiffies g=278549 q=13081) Aug 15 07:07:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 07:07:34 oleg133-client kernel: NMI backtrace for cpu 0 Aug 15 07:07:34 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:07:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:07:34 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:07:34 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:07:34 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 Aug 15 07:07:34 oleg133-client kernel: RAX: 0000000000007f8a RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:07:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:07:34 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:07:34 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:07:34 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:07:34 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:07:34 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:07:34 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:07:34 oleg133-client kernel: Call Trace: Aug 15 07:07:34 oleg133-client kernel: Aug 15 07:07:34 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:07:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x98/0xa2 Aug 15 07:07:34 oleg133-client kernel: ? nmi_cpu_backtrace_handler+0x15/0x40 Aug 15 07:07:34 oleg133-client kernel: ? nmi_handle+0xbd/0x210 Aug 15 07:07:34 oleg133-client kernel: ? default_do_nmi+0x4b/0x1b0 Aug 15 07:07:34 oleg133-client kernel: ? do_nmi+0x280/0x380 Aug 15 07:07:34 oleg133-client kernel: ? end_repeat_nmi+0x16/0x1f Aug 15 07:07:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:07:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:07:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:07:34 oleg133-client kernel: Aug 15 07:07:34 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:07:34 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:07:34 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:07:34 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:07:34 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:07:34 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:07:34 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:07:34 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:07:34 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:07:34 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:07:34 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:07:34 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:07:34 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:07:34 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:07:34 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:07:34 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:07:34 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:07:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:07:34 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:07:34 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:07:34 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:07:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 07:07:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:07:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:07:34 oleg133-client kernel: Call Trace: Aug 15 07:07:34 oleg133-client kernel: Aug 15 07:07:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 07:07:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 07:07:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 07:07:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 07:07:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 07:07:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 07:07:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 07:07:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 07:07:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 07:07:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 07:07:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 07:07:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:07:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 07:07:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:07:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:07:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:07:34 oleg133-client kernel: Aug 15 07:07:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:07:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:07:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 07:07:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 07:07:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:07:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:07:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:07:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:07:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 07:07:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:07:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 07:07:40 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:07:40 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:07:40 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:07:40 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:07:40 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:07:40 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:07:40 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:07:40 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:07:40 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:07:40 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:07:40 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:07:40 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:07:40 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:07:40 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:07:40 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:07:40 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:07:40 oleg133-client kernel: Call Trace: Aug 15 07:07:40 oleg133-client kernel: Aug 15 07:07:40 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:07:40 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:07:40 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:07:40 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:07:40 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:07:40 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:07:40 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:07:40 oleg133-client kernel: Aug 15 07:07:40 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:07:40 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:07:40 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:07:40 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:07:40 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:07:40 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:07:40 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:07:40 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:07:40 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:07:40 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:07:40 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:07:40 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:08:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:08:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:08:00 oleg133-client kernel: Modules linked in: Aug 15 07:08:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:08:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:08:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:08:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:08:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:08:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:08:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:08:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:08:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:08:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:08:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:08:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:08:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:08:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:08:00 oleg133-client kernel: Call Trace: Aug 15 07:08:00 oleg133-client kernel: Aug 15 07:08:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:08:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:08:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:08:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:08:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:08:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:08:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:08:00 oleg133-client kernel: Aug 15 07:08:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:08:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:08:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:08:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:08:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:08:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:08:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:08:00 oleg133-client kernel: lustre(O) Aug 15 07:08:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:08:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:08:00 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:08:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:08:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:08:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:08:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:08:00 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:08:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:08:00 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:08:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:08:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:08:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:08:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:08:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:08:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:08:00 oleg133-client kernel: Call Trace: Aug 15 07:08:00 oleg133-client kernel: Aug 15 07:08:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:08:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:08:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:08:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:08:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:08:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:08:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:08:00 oleg133-client kernel: Aug 15 07:08:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:08:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:08:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:08:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:08:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:08:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:08:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:08:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:08:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:08:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:08:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:08:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:08:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:08:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:08:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:08:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:08:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:08:01 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:08:01 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:08:01 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:08:01 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:08:01 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:08:01 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:08:01 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:08:01 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:08:01 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:08:01 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:08:01 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:08:01 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:08:01 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:08:01 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:08:01 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:08:01 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:08:01 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:08:01 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:08:01 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:08:01 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:08:08 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:08:08 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:08:08 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:08:08 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:08:08 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:08:08 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:08:08 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:08:08 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:08:08 oleg133-client kernel: RAX: 0000000000002d92 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:08:08 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:08:08 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:08:08 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:08:08 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:08:08 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:08:08 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:08:08 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:08:08 oleg133-client kernel: Call Trace: Aug 15 07:08:08 oleg133-client kernel: Aug 15 07:08:08 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:08:08 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:08:08 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:08:08 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:08:08 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:08:08 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:08:08 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:08:08 oleg133-client kernel: Aug 15 07:08:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:08:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:08:08 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:08:08 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:08:08 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:08:08 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:08:08 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:08:08 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:08:08 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:08:08 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:08:08 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:08:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:08:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:08:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 Aug 15 07:08:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:08:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:08:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:08:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:08:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:08:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:08:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:08:28 oleg133-client kernel: RDX: 0000000000007e4b RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:08:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:08:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:08:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:08:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:08:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:08:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:08:28 oleg133-client kernel: Call Trace: Aug 15 07:08:28 oleg133-client kernel: Aug 15 07:08:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:08:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:08:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:08:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:08:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:08:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:08:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:08:28 oleg133-client kernel: Aug 15 07:08:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:08:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:08:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:08:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:08:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:08:28 oleg133-client kernel: squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:08:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:08:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:08:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:08:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:08:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:08:28 oleg133-client kernel: RAX: 0000000000005aee RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:08:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:08:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:08:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:08:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:08:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:08:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:08:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:08:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:08:28 oleg133-client kernel: Call Trace: Aug 15 07:08:28 oleg133-client kernel: Aug 15 07:08:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:08:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:08:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:08:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:08:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:08:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:08:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:08:28 oleg133-client kernel: Aug 15 07:08:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:08:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:08:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:08:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:08:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:08:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:08:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:08:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:08:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:08:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:08:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:08:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:08:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:08:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:08:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:08:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:08:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:08:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:08:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:08:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:08:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:08:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:08:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:08:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:08:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:08:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:08:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:08:29 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:08:29 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:08:29 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:08:29 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:08:29 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:08:29 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:08:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:08:29 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:08:29 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:08:29 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:08:29 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:08:36 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:08:36 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:08:36 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:08:36 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:08:36 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:08:36 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:08:36 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:08:36 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:08:36 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:08:36 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:08:36 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:08:36 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:08:36 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:08:36 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:08:36 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:08:36 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:08:36 oleg133-client kernel: Call Trace: Aug 15 07:08:36 oleg133-client kernel: Aug 15 07:08:36 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:08:36 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:08:36 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:08:36 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:08:36 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:08:36 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:08:36 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:08:36 oleg133-client kernel: Aug 15 07:08:36 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:08:36 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:08:36 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:08:36 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:08:36 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:08:36 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:08:36 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:08:36 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:08:36 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:08:36 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:08:36 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:08:36 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:08:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:08:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:08:56 oleg133-client kernel: Modules linked in: Aug 15 07:08:56 oleg133-client kernel: Modules linked in: Aug 15 07:08:56 oleg133-client kernel: lustre(O) Aug 15 07:08:56 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:08:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:08:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:08:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:08:56 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:08:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:08:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:08:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:08:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:08:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:08:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:08:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:08:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:08:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:08:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:08:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:08:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:08:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:08:56 oleg133-client kernel: Call Trace: Aug 15 07:08:56 oleg133-client kernel: Aug 15 07:08:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:08:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:08:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:08:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:08:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:08:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:08:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:08:56 oleg133-client kernel: Aug 15 07:08:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:08:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:08:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:08:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:08:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:08:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:08:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:08:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:08:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:08:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:08:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:08:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:08:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:08:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:08:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:08:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:08:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:08:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:08:56 oleg133-client kernel: Call Trace: Aug 15 07:08:56 oleg133-client kernel: Aug 15 07:08:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:08:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:08:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:08:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:08:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:08:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:08:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:08:56 oleg133-client kernel: Aug 15 07:08:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:08:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:08:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:08:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:08:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:08:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:08:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:08:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:08:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:08:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:08:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:08:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:08:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:08:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:08:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:08:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:08:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:08:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:08:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:08:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:08:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:08:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:08:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:08:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:08:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:08:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:08:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:08:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:08:57 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:08:57 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:08:57 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:08:57 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:08:57 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:08:57 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:08:57 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:08:57 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:08:57 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:08:57 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:09:04 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:09:04 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:09:04 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:09:04 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:09:04 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:09:04 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:09:04 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:09:04 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:09:04 oleg133-client kernel: RAX: 00000000000073a5 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:09:04 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:09:04 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:09:04 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:09:04 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:09:04 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:09:04 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:09:04 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:09:04 oleg133-client kernel: Call Trace: Aug 15 07:09:04 oleg133-client kernel: Aug 15 07:09:04 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:09:04 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:09:04 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:09:04 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:09:04 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:09:04 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:09:04 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:09:04 oleg133-client kernel: Aug 15 07:09:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:09:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:09:04 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:09:04 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:09:04 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:09:04 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:09:04 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:09:04 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:09:04 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:09:04 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:09:04 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:09:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:09:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:09:24 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 07:09:24 oleg133-client kernel: Modules linked in: Aug 15 07:09:24 oleg133-client kernel: osp(O) ofd(O) Aug 15 07:09:24 oleg133-client kernel: lustre(O) Aug 15 07:09:24 oleg133-client kernel: lod(O) Aug 15 07:09:24 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:09:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:09:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:09:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:09:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:09:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:09:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:09:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:09:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:09:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:09:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:09:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:09:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:09:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:09:24 oleg133-client kernel: Call Trace: Aug 15 07:09:24 oleg133-client kernel: Aug 15 07:09:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:09:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:09:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:09:24 oleg133-client kernel: mdt(O) Aug 15 07:09:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:09:24 oleg133-client kernel: mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:09:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:09:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:09:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:09:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:09:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:09:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:09:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:09:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:09:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:09:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:09:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:09:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:09:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:09:24 oleg133-client kernel: Call Trace: Aug 15 07:09:24 oleg133-client kernel: Aug 15 07:09:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:09:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:09:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:09:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:09:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:09:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:09:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:09:24 oleg133-client kernel: Aug 15 07:09:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:09:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:09:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:09:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:09:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:09:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:09:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:09:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:09:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:09:24 oleg133-client kernel: Aug 15 07:09:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:09:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:09:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:09:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:09:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:09:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:09:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:09:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:09:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:09:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:09:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:09:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:09:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:09:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:09:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:09:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:09:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:09:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:09:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:09:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:09:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:09:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:09:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:09:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:09:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:09:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:09:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:09:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:09:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:09:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:09:32 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:09:32 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:09:32 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:09:32 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:09:32 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:09:32 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:09:32 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:09:32 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:09:32 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:09:32 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:09:32 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:09:32 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:09:32 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:09:32 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:09:32 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:09:32 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:09:32 oleg133-client kernel: Call Trace: Aug 15 07:09:32 oleg133-client kernel: Aug 15 07:09:32 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:09:32 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:09:32 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:09:32 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:09:32 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:09:32 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:09:32 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:09:32 oleg133-client kernel: Aug 15 07:09:32 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:09:32 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:09:32 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:09:32 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:09:32 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:09:32 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:09:32 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:09:32 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:09:32 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:09:32 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:09:32 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:09:32 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:09:32 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 07:09:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:09:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:09:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:09:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:09:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:09:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:09:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:09:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:09:52 oleg133-client kernel: RAX: 0000000000002705 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:09:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:09:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:09:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:09:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:09:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:09:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:09:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:09:52 oleg133-client kernel: Call Trace: Aug 15 07:09:52 oleg133-client kernel: Aug 15 07:09:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:09:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:09:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:09:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:09:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:09:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:09:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:09:52 oleg133-client kernel: Aug 15 07:09:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:09:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:09:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:09:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:09:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:09:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:09:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:09:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:09:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:09:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:09:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:09:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:09:52 oleg133-client kernel: RDX: 0000000000000fae RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:09:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:09:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:09:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:09:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:09:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:09:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:09:52 oleg133-client kernel: Call Trace: Aug 15 07:09:52 oleg133-client kernel: Aug 15 07:09:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:09:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:09:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:09:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:09:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:09:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:09:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:09:52 oleg133-client kernel: Aug 15 07:09:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:09:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:09:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:09:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:09:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:09:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:09:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:09:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:09:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:09:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:09:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:09:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:09:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:09:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:09:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:09:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:09:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:09:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:09:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:09:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:09:53 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:09:53 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:09:53 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:09:53 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:09:53 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:09:53 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:09:53 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:09:53 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:09:53 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:09:53 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:09:53 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:09:53 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:09:53 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:09:53 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:09:53 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:09:53 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:09:53 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:09:53 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:09:53 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:10:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:10:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:10:00 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:10:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:10:00 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:10:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:10:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:10:00 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:10:00 oleg133-client kernel: RAX: 000000000000738f RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:10:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:10:00 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:10:00 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:10:00 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:10:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:10:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:10:00 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:10:00 oleg133-client kernel: Call Trace: Aug 15 07:10:00 oleg133-client kernel: Aug 15 07:10:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:10:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:10:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:10:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:10:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:10:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:10:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:10:00 oleg133-client kernel: Aug 15 07:10:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:10:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:10:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:10:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:10:00 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:10:00 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:10:00 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:10:00 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:10:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:10:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:10:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:10:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:10:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:10:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:10:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:10:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:10:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:10:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:10:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:10:20 oleg133-client kernel: RAX: 0000000000001b25 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:10:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:10:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:10:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:10:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:10:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:10:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:10:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:10:20 oleg133-client kernel: Call Trace: Aug 15 07:10:20 oleg133-client kernel: Aug 15 07:10:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:10:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:10:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:10:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:10:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:10:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:10:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:10:20 oleg133-client kernel: Aug 15 07:10:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:10:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:10:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:10:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:10:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:10:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:10:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:10:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:10:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:10:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:10:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:10:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:10:20 oleg133-client kernel: RDX: 000000000000238f RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:10:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:10:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:10:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:10:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:10:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:10:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:10:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:10:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:10:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:10:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:10:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:10:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:10:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:10:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:10:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:10:20 oleg133-client kernel: Call Trace: Aug 15 07:10:20 oleg133-client kernel: Aug 15 07:10:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:10:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:10:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:10:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:10:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:10:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:10:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:10:20 oleg133-client kernel: Aug 15 07:10:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:10:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:10:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:10:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:10:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:10:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:10:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:10:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:10:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:10:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:10:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:10:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:10:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:10:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:10:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:10:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:10:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:10:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:10:21 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:10:21 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:10:21 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:10:21 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:10:21 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:10:21 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:10:21 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:10:21 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:10:21 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:10:21 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:10:21 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:10:21 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:10:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:10:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:10:28 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:10:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:10:28 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:10:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:10:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:10:28 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:10:28 oleg133-client kernel: RAX: 0000000000007b30 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:10:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:10:28 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:10:28 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:10:28 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:10:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:10:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:10:28 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:10:28 oleg133-client kernel: Call Trace: Aug 15 07:10:28 oleg133-client kernel: Aug 15 07:10:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:10:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:10:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:10:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:10:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:10:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:10:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:10:28 oleg133-client kernel: Aug 15 07:10:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:10:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:10:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:10:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:10:28 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:10:28 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:10:28 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:10:28 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:10:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:10:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:10:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:10:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 07:10:34 oleg133-client kernel: rcu: #0111-....: (1760439 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=501110 Aug 15 07:10:34 oleg133-client kernel: #011(t=2580057 jiffies g=278549 q=13897) Aug 15 07:10:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 07:10:34 oleg133-client kernel: NMI backtrace for cpu 0 Aug 15 07:10:34 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:10:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:10:34 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:10:34 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:10:34 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 Aug 15 07:10:34 oleg133-client kernel: RAX: 0000000000007ebe RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:10:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:10:34 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:10:34 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:10:34 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:10:34 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:10:34 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:10:34 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:10:34 oleg133-client kernel: Call Trace: Aug 15 07:10:34 oleg133-client kernel: Aug 15 07:10:34 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:10:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x98/0xa2 Aug 15 07:10:34 oleg133-client kernel: ? nmi_cpu_backtrace_handler+0x15/0x40 Aug 15 07:10:34 oleg133-client kernel: ? nmi_handle+0xbd/0x210 Aug 15 07:10:34 oleg133-client kernel: ? default_do_nmi+0x4b/0x1b0 Aug 15 07:10:34 oleg133-client kernel: ? do_nmi+0x280/0x380 Aug 15 07:10:34 oleg133-client kernel: ? end_repeat_nmi+0x16/0x1f Aug 15 07:10:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:10:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:10:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:10:34 oleg133-client kernel: Aug 15 07:10:34 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:10:34 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:10:34 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:10:34 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:10:34 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:10:34 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:10:34 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:10:34 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:10:34 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:10:34 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:10:34 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:10:34 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:10:34 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:10:34 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:10:34 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:10:34 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:10:34 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:10:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:10:34 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:10:34 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:10:34 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:10:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 07:10:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:10:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:10:34 oleg133-client kernel: Call Trace: Aug 15 07:10:34 oleg133-client kernel: Aug 15 07:10:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 07:10:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 07:10:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 07:10:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 07:10:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 07:10:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 07:10:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 07:10:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 07:10:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 07:10:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 07:10:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 07:10:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:10:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 07:10:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:10:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:10:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:10:34 oleg133-client kernel: Aug 15 07:10:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:10:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:10:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 07:10:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 07:10:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:10:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:10:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:10:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:10:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 07:10:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:10:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 07:10:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:10:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:10:56 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:10:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:10:56 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:10:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:10:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:10:56 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:10:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:10:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:10:56 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:10:56 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:10:56 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:10:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:10:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:10:56 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:10:56 oleg133-client kernel: Call Trace: Aug 15 07:10:56 oleg133-client kernel: Aug 15 07:10:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:10:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:10:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:10:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:10:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:10:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:10:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:10:56 oleg133-client kernel: Aug 15 07:10:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:10:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:10:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:10:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:10:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:10:56 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:10:56 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:10:56 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:10:56 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:10:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:10:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:10:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:11:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:11:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:11:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:11:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:11:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:11:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:11:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:11:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:11:00 oleg133-client kernel: RAX: 0000000000003b8d RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:11:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:11:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:11:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:11:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:11:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:11:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:11:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:11:00 oleg133-client kernel: Call Trace: Aug 15 07:11:00 oleg133-client kernel: Aug 15 07:11:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:11:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:11:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:11:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:11:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:11:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:11:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:11:00 oleg133-client kernel: Aug 15 07:11:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:11:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:11:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:11:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:11:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:11:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:11:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:11:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:11:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:11:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:11:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:11:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:11:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:11:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:11:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:11:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:11:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:11:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:11:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:11:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:11:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:11:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:11:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:11:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:11:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:11:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:11:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:11:00 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:11:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:11:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:11:00 oleg133-client kernel: RDX: 0000000000003fc0 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:11:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:11:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:11:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:11:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:11:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:11:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:11:00 oleg133-client kernel: Call Trace: Aug 15 07:11:00 oleg133-client kernel: Aug 15 07:11:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:11:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:11:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:11:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:11:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:11:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:11:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:11:00 oleg133-client kernel: Aug 15 07:11:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:11:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:11:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:11:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:11:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:11:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:11:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:11:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:11:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:11:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:11:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:11:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:11:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:11:24 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:11:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:11:24 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:11:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:11:24 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:11:24 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:11:24 oleg133-client kernel: RAX: 0000000000007e96 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:11:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:11:24 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:11:24 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:11:24 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:11:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:11:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:11:24 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:11:24 oleg133-client kernel: Call Trace: Aug 15 07:11:24 oleg133-client kernel: Aug 15 07:11:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:11:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:11:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:11:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:11:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:11:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:11:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:11:24 oleg133-client kernel: Aug 15 07:11:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:11:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:11:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:11:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:11:24 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:11:24 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:11:24 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:11:24 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:11:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:11:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:11:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:11:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:11:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:11:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:11:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:11:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:11:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:11:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:11:28 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:11:28 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:11:28 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:11:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:11:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:11:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:11:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:11:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:11:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:11:28 oleg133-client kernel: Call Trace: Aug 15 07:11:28 oleg133-client kernel: Aug 15 07:11:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:11:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:11:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:11:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:11:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:11:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:11:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:11:28 oleg133-client kernel: Aug 15 07:11:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:11:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:11:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:11:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:11:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:11:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:11:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:11:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:11:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:11:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:11:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:11:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:11:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:11:28 oleg133-client kernel: RDX: 0000000000007ebd RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:11:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:11:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:11:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:11:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:11:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:11:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:11:28 oleg133-client kernel: Call Trace: Aug 15 07:11:28 oleg133-client kernel: Aug 15 07:11:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:11:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:11:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:11:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:11:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:11:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:11:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:11:28 oleg133-client kernel: Aug 15 07:11:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:11:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:11:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:11:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:11:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:11:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:11:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:11:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:11:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:11:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:11:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:11:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:11:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:11:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:11:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:11:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:11:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:11:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:11:29 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:11:29 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:11:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:11:29 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:11:29 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:11:29 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:11:29 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:11:29 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:11:29 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:11:29 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:11:29 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:11:29 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:11:29 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:11:29 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:11:29 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:11:29 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:11:29 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:11:29 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:11:29 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:11:29 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:11:29 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:11:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:11:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:11:52 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:11:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:11:52 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:11:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:11:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:11:52 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:11:52 oleg133-client kernel: RAX: 0000000000005a73 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:11:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:11:52 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:11:52 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:11:52 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:11:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:11:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:11:52 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:11:52 oleg133-client kernel: Call Trace: Aug 15 07:11:52 oleg133-client kernel: Aug 15 07:11:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:11:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:11:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:11:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:11:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:11:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:11:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:11:52 oleg133-client kernel: Aug 15 07:11:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:11:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:11:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:11:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:11:52 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:11:52 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:11:52 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:11:52 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:11:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:11:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:11:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:11:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:11:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:11:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:11:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:11:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:11:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:11:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:11:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:11:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:11:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:11:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:11:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:11:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:11:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:11:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:11:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:11:56 oleg133-client kernel: Call Trace: Aug 15 07:11:56 oleg133-client kernel: Aug 15 07:11:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:11:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:11:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:11:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:11:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:11:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:11:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:11:56 oleg133-client kernel: Aug 15 07:11:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:11:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:11:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:11:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:11:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:11:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:11:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:11:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:11:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:11:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:11:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:11:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:11:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:11:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:11:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:11:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:11:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:11:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:11:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:11:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:11:56 oleg133-client kernel: Call Trace: Aug 15 07:11:56 oleg133-client kernel: Aug 15 07:11:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:11:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:11:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:11:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:11:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:11:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:11:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:11:56 oleg133-client kernel: Aug 15 07:11:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:11:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:11:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:11:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:11:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:11:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:11:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:11:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:11:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:11:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:11:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:11:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:11:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:11:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:11:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:11:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:11:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:11:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:11:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:11:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:11:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:11:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:11:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:11:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:11:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:11:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:11:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:11:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:11:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:11:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:12:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:12:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:12:20 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:12:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:12:20 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:12:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:12:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:12:20 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:12:20 oleg133-client kernel: RAX: 0000000000007585 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:12:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:12:20 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:12:20 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:12:20 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:12:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:12:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:12:20 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:12:20 oleg133-client kernel: Call Trace: Aug 15 07:12:20 oleg133-client kernel: Aug 15 07:12:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:12:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:12:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:12:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:12:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:12:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:12:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:12:20 oleg133-client kernel: Aug 15 07:12:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:12:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:12:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:12:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:12:20 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:12:20 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:12:20 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:12:20 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:12:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:12:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:12:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:12:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:12:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:12:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:12:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:12:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:12:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:12:24 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:12:24 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:12:24 oleg133-client kernel: RAX: 0000000000007114 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:12:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:12:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:12:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:12:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:12:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:12:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:12:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:12:24 oleg133-client kernel: Call Trace: Aug 15 07:12:24 oleg133-client kernel: Aug 15 07:12:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:12:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:12:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:12:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:12:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:12:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:12:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:12:24 oleg133-client kernel: Aug 15 07:12:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:12:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:12:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:12:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:12:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:12:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:12:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:12:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:12:24 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:12:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:12:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:12:24 oleg133-client kernel: RDX: 000000000000466d RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:12:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:12:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:12:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:12:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:12:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:12:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:12:24 oleg133-client kernel: Call Trace: Aug 15 07:12:24 oleg133-client kernel: Aug 15 07:12:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:12:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:12:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:12:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:12:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:12:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:12:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:12:24 oleg133-client kernel: Aug 15 07:12:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:12:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:12:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:12:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:12:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:12:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:12:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:12:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:12:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:12:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:12:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:12:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:12:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:12:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:12:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:12:25 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:12:25 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:12:25 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:12:25 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:12:25 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:12:25 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:12:25 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:12:25 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:12:25 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:12:25 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:12:25 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:12:25 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:12:25 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:12:25 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:12:25 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:12:48 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:12:48 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:12:48 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:12:48 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:12:48 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:12:48 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:12:48 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:12:48 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:12:48 oleg133-client kernel: RAX: 000000000000590c RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:12:48 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:12:48 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:12:48 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:12:48 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:12:48 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:12:48 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:12:48 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:12:48 oleg133-client kernel: Call Trace: Aug 15 07:12:48 oleg133-client kernel: Aug 15 07:12:48 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:12:48 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:12:48 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:12:48 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:12:48 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:12:48 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:12:48 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:12:48 oleg133-client kernel: Aug 15 07:12:48 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:12:48 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:12:48 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:12:48 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:12:48 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:12:48 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:12:48 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:12:48 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:12:48 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:12:48 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:12:48 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:12:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:12:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:12:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) Aug 15 07:12:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:12:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:12:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:12:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:12:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:12:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:12:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:12:52 oleg133-client kernel: RDX: 00000000000019b9 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:12:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:12:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:12:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:12:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:12:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:12:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:12:52 oleg133-client kernel: Call Trace: Aug 15 07:12:52 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:12:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:12:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:12:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:12:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:12:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:12:52 oleg133-client kernel: RAX: 000000000000450b RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:12:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:12:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:12:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:12:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:12:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:12:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:12:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:12:52 oleg133-client kernel: Call Trace: Aug 15 07:12:52 oleg133-client kernel: Aug 15 07:12:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:12:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:12:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:12:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:12:52 oleg133-client kernel: Aug 15 07:12:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:12:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:12:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:12:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:12:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:12:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:12:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:12:52 oleg133-client kernel: Aug 15 07:12:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:12:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:12:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:12:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:12:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:12:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:12:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:12:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:12:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:12:52 oleg133-client kernel: Aug 15 07:12:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:12:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:12:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:12:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:12:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:12:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:12:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:12:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:12:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:12:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:12:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:12:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:12:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:12:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:12:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:12:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:12:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:12:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:12:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:12:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:12:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:12:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:12:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:12:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:12:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:12:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:12:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:12:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:13:16 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:13:16 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:13:16 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:13:16 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:13:16 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:13:16 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:13:16 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:13:16 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:13:16 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:13:16 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:13:16 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:13:16 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:13:16 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:13:16 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:13:16 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:13:16 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:13:16 oleg133-client kernel: Call Trace: Aug 15 07:13:16 oleg133-client kernel: Aug 15 07:13:16 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:13:16 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:13:16 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:13:16 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:13:16 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:13:16 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:13:16 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:13:16 oleg133-client kernel: Aug 15 07:13:16 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:13:16 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:13:16 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:13:16 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:13:16 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:13:16 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:13:16 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:13:16 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:13:16 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:13:16 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:13:16 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:13:16 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:13:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:13:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:13:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:13:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:13:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:13:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:13:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:13:20 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:13:20 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:13:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:13:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:13:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:13:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:13:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:13:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:13:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:13:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:13:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:13:20 oleg133-client kernel: Call Trace: Aug 15 07:13:20 oleg133-client kernel: Aug 15 07:13:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:13:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:13:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:13:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:13:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:13:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:13:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:13:20 oleg133-client kernel: Aug 15 07:13:20 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:13:20 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:13:20 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:13:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:13:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:13:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:13:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:13:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:13:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:13:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:13:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:13:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:13:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:13:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:13:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:13:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:13:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:13:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:13:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:13:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:13:21 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:13:21 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:13:21 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 Aug 15 07:13:21 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:13:21 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:13:21 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:13:21 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:13:21 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:13:21 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:13:21 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:13:21 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:13:21 oleg133-client kernel: Call Trace: Aug 15 07:13:21 oleg133-client kernel: Aug 15 07:13:21 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:13:21 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:13:21 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:13:21 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:13:21 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:13:21 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:13:21 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:13:21 oleg133-client kernel: Aug 15 07:13:21 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:13:21 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:13:21 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:13:21 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:13:21 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:13:21 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:13:21 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:13:21 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:13:21 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:13:21 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:13:21 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:13:21 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:13:21 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:13:21 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:13:21 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:13:21 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:13:21 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:13:21 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:13:21 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:13:21 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:13:21 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:13:21 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:13:21 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:13:21 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:13:21 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:13:21 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:13:21 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:13:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 07:13:34 oleg133-client kernel: rcu: #0111-....: (1882726 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=536125 Aug 15 07:13:34 oleg133-client kernel: #011(t=2760060 jiffies g=278549 q=14570) Aug 15 07:13:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 07:13:34 oleg133-client kernel: NMI backtrace for cpu 0 Aug 15 07:13:34 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:13:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:13:34 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:13:34 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:13:34 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 Aug 15 07:13:34 oleg133-client kernel: RAX: 0000000000003c23 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:13:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:13:34 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:13:34 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:13:34 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:13:34 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:13:34 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:13:34 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:13:34 oleg133-client kernel: Call Trace: Aug 15 07:13:34 oleg133-client kernel: Aug 15 07:13:34 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:13:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x98/0xa2 Aug 15 07:13:34 oleg133-client kernel: ? nmi_cpu_backtrace_handler+0x15/0x40 Aug 15 07:13:34 oleg133-client kernel: ? nmi_handle+0xbd/0x210 Aug 15 07:13:34 oleg133-client kernel: ? default_do_nmi+0x4b/0x1b0 Aug 15 07:13:34 oleg133-client kernel: ? do_nmi+0x280/0x380 Aug 15 07:13:34 oleg133-client kernel: ? end_repeat_nmi+0x16/0x1f Aug 15 07:13:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:13:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:13:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:13:34 oleg133-client kernel: Aug 15 07:13:34 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:13:34 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:13:34 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:13:34 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:13:34 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:13:34 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:13:34 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:13:34 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:13:34 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:13:34 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:13:34 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:13:34 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:13:34 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:13:34 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:13:34 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:13:34 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:13:34 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:13:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:13:34 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:13:34 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:13:34 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:13:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 07:13:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:13:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:13:34 oleg133-client kernel: Call Trace: Aug 15 07:13:34 oleg133-client kernel: Aug 15 07:13:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 07:13:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 07:13:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 07:13:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 07:13:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 07:13:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 07:13:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 07:13:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 07:13:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 07:13:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 07:13:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 07:13:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:13:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 07:13:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:13:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:13:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:13:34 oleg133-client kernel: Aug 15 07:13:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:13:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:13:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 07:13:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 07:13:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:13:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:13:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:13:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:13:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 07:13:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:13:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 07:13:44 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:13:44 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:13:44 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:13:44 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:13:44 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:13:44 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:13:44 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:13:44 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:13:44 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:13:44 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:13:44 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:13:44 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:13:44 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:13:44 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:13:44 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:13:44 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:13:44 oleg133-client kernel: Call Trace: Aug 15 07:13:44 oleg133-client kernel: Aug 15 07:13:44 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:13:44 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:13:44 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:13:44 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:13:44 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:13:44 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:13:44 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:13:44 oleg133-client kernel: Aug 15 07:13:44 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:13:44 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:13:44 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:13:44 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:13:44 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:13:44 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:13:44 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:13:44 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:13:44 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:13:44 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:13:44 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:13:44 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:14:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:14:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:14:00 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 07:14:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:14:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:14:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:14:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:14:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:14:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:14:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:14:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:14:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:14:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:14:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:14:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:14:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:14:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:14:00 oleg133-client kernel: Call Trace: Aug 15 07:14:00 oleg133-client kernel: Aug 15 07:14:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:14:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:14:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:14:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:14:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:14:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:14:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:14:00 oleg133-client kernel: Aug 15 07:14:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:14:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:14:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:14:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:14:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:14:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:14:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:14:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:14:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:14:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:14:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:14:00 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 Aug 15 07:14:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:14:00 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:14:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:14:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:14:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:14:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:14:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:14:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:14:00 oleg133-client kernel: Call Trace: Aug 15 07:14:00 oleg133-client kernel: Aug 15 07:14:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:14:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:14:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:14:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:14:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:14:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:14:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:14:00 oleg133-client kernel: Aug 15 07:14:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:14:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:14:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:14:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:14:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:14:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:14:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:14:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:14:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:14:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:14:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:14:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:14:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:14:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:14:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:14:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:14:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:14:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:14:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:14:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:14:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:14:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:14:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:14:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:14:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:14:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:14:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:14:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:14:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:14:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:14:12 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:14:12 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:14:12 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:14:12 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:14:12 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:14:12 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:14:12 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:14:12 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:14:12 oleg133-client kernel: RAX: 000000000000420b RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:14:12 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:14:12 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:14:12 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:14:12 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:14:12 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:14:12 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:14:12 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:14:12 oleg133-client kernel: Call Trace: Aug 15 07:14:12 oleg133-client kernel: Aug 15 07:14:12 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:14:12 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:14:12 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:14:12 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:14:12 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:14:12 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:14:12 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:14:12 oleg133-client kernel: Aug 15 07:14:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:14:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:14:12 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:14:12 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:14:12 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:14:12 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:14:12 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:14:12 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:14:12 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:14:12 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:14:12 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:14:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:14:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:14:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:14:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:14:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:14:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:14:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:14:28 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:14:28 oleg133-client kernel: RAX: 0000000000007f0a RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:14:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:14:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:14:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:14:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:14:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:14:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:14:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:14:28 oleg133-client kernel: Call Trace: Aug 15 07:14:28 oleg133-client kernel: Aug 15 07:14:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:14:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:14:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:14:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:14:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:14:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:14:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:14:28 oleg133-client kernel: Aug 15 07:14:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:14:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:14:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:14:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:14:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:14:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:14:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:14:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:14:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:14:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:14:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:14:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:14:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:14:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:14:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:14:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:14:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:14:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:14:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:14:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:14:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:14:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:14:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:14:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:14:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:14:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:14:29 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:14:29 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:14:29 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:14:29 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:14:29 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:14:29 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:14:29 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:14:29 oleg133-client kernel: RDX: 000000000000601a RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:14:29 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:14:29 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:14:29 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:14:29 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:14:29 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:14:29 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:14:29 oleg133-client kernel: Call Trace: Aug 15 07:14:29 oleg133-client kernel: Aug 15 07:14:29 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:14:29 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:14:29 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:14:29 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:14:29 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:14:29 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:14:29 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:14:29 oleg133-client kernel: Aug 15 07:14:29 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:14:29 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:14:29 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:14:29 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:14:29 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:14:29 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:14:29 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:14:29 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:14:29 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:14:29 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:14:29 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:14:29 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:14:29 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:14:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:14:29 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:14:29 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:14:29 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:14:29 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:14:29 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 07:14:40 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:14:40 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:14:40 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:14:40 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:14:40 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:14:40 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:14:40 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:14:40 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:14:40 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:14:40 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:14:40 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:14:40 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:14:40 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:14:40 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:14:40 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:14:40 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:14:40 oleg133-client kernel: Call Trace: Aug 15 07:14:40 oleg133-client kernel: Aug 15 07:14:40 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:14:40 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:14:40 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:14:40 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:14:40 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:14:40 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:14:40 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:14:40 oleg133-client kernel: Aug 15 07:14:40 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:14:40 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:14:40 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:14:40 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:14:40 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:14:40 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:14:40 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:14:40 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:14:40 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:14:40 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:14:40 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:14:40 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:14:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:14:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:14:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:14:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:14:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:14:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:14:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:14:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:14:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:14:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:14:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:14:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:14:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:14:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:14:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:14:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:14:56 oleg133-client kernel: Call Trace: Aug 15 07:14:56 oleg133-client kernel: Aug 15 07:14:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:14:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:14:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:14:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:14:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:14:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:14:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:14:56 oleg133-client kernel: Aug 15 07:14:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:14:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:14:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:14:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:14:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:14:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:14:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:14:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:14:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:14:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:14:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:14:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:14:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:14:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:14:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:14:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:14:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:14:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:14:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:14:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:14:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:14:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:14:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:14:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:14:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:14:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:14:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:14:57 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:14:57 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:14:57 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:14:57 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:14:57 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:14:57 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:14:57 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:14:57 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:14:57 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:14:57 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:14:57 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:14:57 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:14:57 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:14:57 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:14:57 oleg133-client kernel: Call Trace: Aug 15 07:14:57 oleg133-client kernel: Aug 15 07:14:57 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:14:57 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:14:57 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:14:57 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:14:57 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:14:57 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:14:57 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:14:57 oleg133-client kernel: Aug 15 07:14:57 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:14:57 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:14:57 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:14:57 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:14:57 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:14:57 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:14:57 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:14:57 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:14:57 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:14:57 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:14:57 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:14:57 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:14:57 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:14:57 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:14:57 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:14:57 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:14:57 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:14:57 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:14:57 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:15:08 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:15:08 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:15:08 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:15:08 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:15:08 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:15:08 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:15:08 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:15:08 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 Aug 15 07:15:08 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:15:08 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:15:08 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:15:08 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:15:08 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:15:08 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:15:08 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:15:08 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:15:08 oleg133-client kernel: Call Trace: Aug 15 07:15:08 oleg133-client kernel: Aug 15 07:15:08 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:15:08 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:15:08 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:15:08 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:15:08 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:15:08 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:15:08 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:15:08 oleg133-client kernel: Aug 15 07:15:08 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:15:08 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:15:08 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:15:08 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:15:08 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:15:08 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:15:08 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:15:08 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:15:08 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:15:08 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:15:08 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:15:08 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:15:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:15:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:15:24 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 07:15:24 oleg133-client kernel: Modules linked in: Aug 15 07:15:24 oleg133-client kernel: osp(O) ofd(O) Aug 15 07:15:24 oleg133-client kernel: lustre(O) Aug 15 07:15:24 oleg133-client kernel: lod(O) Aug 15 07:15:24 oleg133-client kernel: osp(O) Aug 15 07:15:24 oleg133-client kernel: mdt(O) Aug 15 07:15:24 oleg133-client kernel: ofd(O) Aug 15 07:15:24 oleg133-client kernel: mdd(O) Aug 15 07:15:24 oleg133-client kernel: lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:15:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:15:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:15:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:15:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:15:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:15:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:15:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:15:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:15:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:15:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:15:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:15:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:15:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:15:24 oleg133-client kernel: Call Trace: Aug 15 07:15:24 oleg133-client kernel: Aug 15 07:15:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:15:24 oleg133-client kernel: mgs(O) Aug 15 07:15:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:15:24 oleg133-client kernel: lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:15:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:15:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:15:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:15:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:15:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:15:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:15:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:15:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:15:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:15:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:15:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:15:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:15:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:15:24 oleg133-client kernel: Call Trace: Aug 15 07:15:24 oleg133-client kernel: Aug 15 07:15:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:15:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:15:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:15:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:15:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:15:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:15:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:15:24 oleg133-client kernel: Aug 15 07:15:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:15:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:15:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:15:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:15:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:15:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:15:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:15:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:15:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:15:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:15:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:15:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:15:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:15:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:15:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:15:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:15:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:15:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:15:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:15:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:15:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:15:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:15:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:15:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:15:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:15:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:15:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:15:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:15:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:15:24 oleg133-client kernel: Aug 15 07:15:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:15:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:15:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:15:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:15:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:15:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:15:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:15:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:15:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:15:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:15:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:15:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:15:36 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:15:36 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:15:36 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:15:36 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:15:36 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:15:36 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:15:36 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:15:36 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:15:36 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:15:36 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:15:36 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:15:36 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:15:36 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:15:36 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:15:36 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:15:36 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:15:36 oleg133-client kernel: Call Trace: Aug 15 07:15:36 oleg133-client kernel: Aug 15 07:15:36 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:15:36 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:15:36 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:15:36 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:15:36 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:15:36 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:15:36 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:15:36 oleg133-client kernel: Aug 15 07:15:36 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:15:36 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:15:36 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:15:36 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:15:36 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:15:36 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:15:36 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:15:36 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:15:36 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:15:36 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:15:36 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:15:36 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:15:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:15:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:15:52 oleg133-client kernel: Modules linked in: Aug 15 07:15:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:15:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:15:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:15:52 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:15:52 oleg133-client kernel: lustre(O) Aug 15 07:15:52 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:15:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:15:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:15:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:15:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:15:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:15:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:15:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:15:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:15:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:15:52 oleg133-client kernel: Call Trace: Aug 15 07:15:52 oleg133-client kernel: Aug 15 07:15:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:15:52 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:15:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:15:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:15:52 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:15:52 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:15:52 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:15:52 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:15:52 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:15:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:15:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:15:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:15:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:15:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:15:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:15:52 oleg133-client kernel: Call Trace: Aug 15 07:15:52 oleg133-client kernel: Aug 15 07:15:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:15:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:15:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:15:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:15:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:15:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:15:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:15:52 oleg133-client kernel: Aug 15 07:15:52 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:15:52 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:15:52 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:15:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:15:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:15:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:15:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:15:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:15:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:15:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:15:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:15:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:15:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:15:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:15:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:15:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:15:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:15:52 oleg133-client kernel: Aug 15 07:15:52 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:15:52 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:15:52 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:15:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:15:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:15:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:15:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:15:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:15:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:15:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:15:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:15:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:15:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:15:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:15:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:15:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:15:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:15:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:15:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:15:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:15:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:15:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:15:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:15:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:15:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:16:04 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:16:04 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:16:04 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:16:04 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:16:04 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:16:04 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:16:04 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:16:04 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:16:04 oleg133-client kernel: RAX: 0000000000007714 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:16:04 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:16:04 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:16:04 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:16:04 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:16:04 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:16:04 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:16:04 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:16:04 oleg133-client kernel: Call Trace: Aug 15 07:16:04 oleg133-client kernel: Aug 15 07:16:04 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:16:04 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:16:04 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:16:04 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:16:04 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:16:04 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:16:04 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:16:04 oleg133-client kernel: Aug 15 07:16:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:16:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:16:04 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:16:04 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:16:04 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:16:04 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:16:04 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:16:04 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:16:04 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:16:04 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:16:04 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:16:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:16:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:16:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:16:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:16:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:16:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:16:20 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:16:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:16:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:16:20 oleg133-client kernel: RDX: 0000000000006b4d RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:16:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:16:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:16:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:16:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:16:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:16:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:16:20 oleg133-client kernel: Call Trace: Aug 15 07:16:20 oleg133-client kernel: Aug 15 07:16:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:16:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:16:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:16:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:16:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:16:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:16:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:16:20 oleg133-client kernel: Aug 15 07:16:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:16:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:16:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:16:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:16:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:16:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:16:20 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 07:16:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:16:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:16:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:16:20 oleg133-client kernel: osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:16:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:16:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:16:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:16:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:16:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:16:20 oleg133-client kernel: RAX: 000000000000560d RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:16:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:16:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:16:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:16:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:16:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:16:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:16:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:16:20 oleg133-client kernel: Call Trace: Aug 15 07:16:20 oleg133-client kernel: Aug 15 07:16:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:16:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:16:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:16:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:16:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:16:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:16:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:16:20 oleg133-client kernel: Aug 15 07:16:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:16:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:16:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:16:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:16:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:16:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:16:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:16:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:16:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:16:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:16:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:16:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:16:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:16:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:16:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:16:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:16:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:16:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:16:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:16:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:16:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:16:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:16:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:16:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:16:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:16:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:16:21 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:16:21 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:16:21 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:16:21 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:16:21 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:16:21 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:16:21 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:16:21 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:16:21 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:16:32 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:16:32 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:16:32 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:16:32 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:16:32 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:16:32 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:16:32 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:16:32 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:16:32 oleg133-client kernel: RAX: 000000000000705b RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:16:32 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:16:32 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:16:32 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:16:32 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:16:32 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:16:32 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:16:32 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:16:32 oleg133-client kernel: Call Trace: Aug 15 07:16:32 oleg133-client kernel: Aug 15 07:16:32 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:16:32 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:16:32 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:16:32 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:16:32 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:16:32 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:16:32 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:16:32 oleg133-client kernel: Aug 15 07:16:32 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:16:32 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:16:32 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:16:32 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:16:32 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:16:32 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:16:32 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:16:32 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:16:32 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:16:32 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:16:32 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:16:34 oleg133-client kernel: rcu: INFO: rcu_sched detected stalls on CPUs/tasks: Aug 15 07:16:34 oleg133-client kernel: rcu: #0110-....: (1970205 ticks this GP) idle=c12/1/0x4000000000000000 softirq=132993/132993 fqs=569910 Aug 15 07:16:34 oleg133-client kernel: rcu: #0111-....: (1995369 ticks this GP) idle=fe2/1/0x4000000000000000 softirq=133713/133713 fqs=569911 Aug 15 07:16:34 oleg133-client kernel: #011(detected by 3, t=2940105 jiffies, g=278549, q=15389) Aug 15 07:16:34 oleg133-client kernel: Sending NMI from CPU 3 to CPUs 0: Aug 15 07:16:34 oleg133-client kernel: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 Aug 15 07:16:34 oleg133-client kernel: Sending NMI from CPU 3 to CPUs 1: Aug 15 07:16:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 07:16:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:16:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:16:34 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:16:34 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:16:34 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 Aug 15 07:16:34 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:16:34 oleg133-client kernel: RDX: 0000000000007dfc RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:16:34 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:16:34 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:16:34 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:16:34 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:16:34 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:16:34 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:16:34 oleg133-client kernel: Call Trace: Aug 15 07:16:34 oleg133-client kernel: Aug 15 07:16:34 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:16:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x98/0xa2 Aug 15 07:16:34 oleg133-client kernel: ? nmi_cpu_backtrace_handler+0x15/0x40 Aug 15 07:16:34 oleg133-client kernel: ? nmi_handle+0xbd/0x210 Aug 15 07:16:34 oleg133-client kernel: ? default_do_nmi+0x4b/0x1b0 Aug 15 07:16:34 oleg133-client kernel: ? do_nmi+0x280/0x380 Aug 15 07:16:34 oleg133-client kernel: ? end_repeat_nmi+0x16/0x1f Aug 15 07:16:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:16:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:16:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:16:34 oleg133-client kernel: Aug 15 07:16:34 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:16:34 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:16:34 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:16:34 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:16:34 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:16:34 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:16:34 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:16:34 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:16:34 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:16:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:16:34 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:16:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:16:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:16:34 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:16:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:16:34 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:17:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:17:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:17:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:17:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:17:00 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:17:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:17:00 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:17:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:17:00 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:17:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:17:00 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:17:00 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:17:00 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:17:00 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:17:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:17:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:17:00 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:17:00 oleg133-client kernel: Call Trace: Aug 15 07:17:00 oleg133-client kernel: Aug 15 07:17:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:17:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:17:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:17:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:17:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:17:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:17:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:17:00 oleg133-client kernel: Aug 15 07:17:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:17:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:17:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:17:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:17:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:17:00 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:17:00 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:17:00 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:17:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:17:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:17:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:17:00 oleg133-client kernel: Modules linked in: Aug 15 07:17:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:17:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:17:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:17:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:17:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:17:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:17:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:17:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:17:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:17:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:17:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:17:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:17:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:17:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:17:00 oleg133-client kernel: Call Trace: Aug 15 07:17:00 oleg133-client kernel: Aug 15 07:17:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:17:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:17:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:17:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:17:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:17:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:17:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:17:00 oleg133-client kernel: Aug 15 07:17:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:17:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:17:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:17:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:17:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:17:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:17:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:17:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:17:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:17:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:17:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:17:00 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:17:00 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:17:00 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:17:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:17:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:17:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:17:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:17:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:17:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:17:00 oleg133-client kernel: Call Trace: Aug 15 07:17:00 oleg133-client kernel: Aug 15 07:17:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:17:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:17:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:17:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:17:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:17:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:17:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:17:00 oleg133-client kernel: Aug 15 07:17:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:17:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:17:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:17:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:17:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:17:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:17:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:17:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:17:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:17:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:17:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:17:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:17:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:17:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:17:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:17:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:17:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:17:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:17:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:17:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:17:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:17:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:17:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:17:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:17:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:17:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:17:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:17:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:17:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:17:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:17:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:17:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:17:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:17:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:17:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:17:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:17:28 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:17:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:17:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:17:28 oleg133-client kernel: RDX: 0000000000007d0a RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:17:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:17:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:17:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:17:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:17:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:17:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:17:28 oleg133-client kernel: Call Trace: Aug 15 07:17:28 oleg133-client kernel: Aug 15 07:17:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:17:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:17:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:17:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:17:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:17:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:17:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:17:28 oleg133-client kernel: Aug 15 07:17:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:17:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:17:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:17:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:17:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:17:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:17:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:17:28 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:17:28 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:17:28 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:17:28 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:17:28 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:17:28 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:17:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:17:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) Aug 15 07:17:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:17:28 oleg133-client kernel: mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:17:28 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:17:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:17:28 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:17:28 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:17:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:17:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:17:28 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:17:28 oleg133-client kernel: RAX: 0000000000006453 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:17:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:17:28 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:17:28 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:17:28 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:17:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:17:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:17:28 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:17:28 oleg133-client kernel: Call Trace: Aug 15 07:17:28 oleg133-client kernel: Aug 15 07:17:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:17:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:17:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:17:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:17:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:17:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:17:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:17:28 oleg133-client kernel: Aug 15 07:17:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:17:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:17:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:17:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:17:28 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:17:28 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:17:28 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:17:28 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:17:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:17:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:17:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:17:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:17:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:17:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:17:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:17:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:17:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:17:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:17:29 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:17:29 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:17:29 oleg133-client kernel: RAX: 0000000000005e09 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:17:29 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:17:29 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:17:29 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:17:29 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:17:29 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:17:29 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:17:29 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:17:29 oleg133-client kernel: Call Trace: Aug 15 07:17:29 oleg133-client kernel: Aug 15 07:17:29 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:17:29 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:17:29 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:17:29 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:17:29 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:17:29 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:17:29 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:17:29 oleg133-client kernel: Aug 15 07:17:29 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:17:29 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:17:29 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:17:29 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:17:29 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:17:29 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:17:29 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:17:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:17:29 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:17:29 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:17:29 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:17:29 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:17:29 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:17:29 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:17:29 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:17:29 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:17:29 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:17:29 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:17:29 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:17:29 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:17:29 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:17:29 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:17:29 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:17:29 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:17:29 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:17:29 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:17:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:17:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:17:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:17:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:17:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:17:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:17:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:17:56 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:17:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:17:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:17:56 oleg133-client kernel: RDX: 00000000000003bd RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:17:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:17:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:17:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:17:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:17:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:17:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:17:56 oleg133-client kernel: Call Trace: Aug 15 07:17:56 oleg133-client kernel: Aug 15 07:17:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:17:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:17:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:17:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:17:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:17:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:17:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:17:56 oleg133-client kernel: Aug 15 07:17:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:17:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:17:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:17:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:17:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:17:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:17:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:17:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:17:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:17:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:17:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) Aug 15 07:17:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:17:56 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:17:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:17:56 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:17:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:17:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:17:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:17:56 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:17:56 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:17:56 oleg133-client kernel: RAX: 00000000000022ca RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:17:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:17:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:17:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:17:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:17:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:17:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:17:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:17:56 oleg133-client kernel: Call Trace: Aug 15 07:17:56 oleg133-client kernel: Aug 15 07:17:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:17:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:17:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:17:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:17:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:17:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:17:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:17:56 oleg133-client kernel: Aug 15 07:17:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:17:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:17:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:17:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:17:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:17:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:17:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:17:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:17:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:17:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:17:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:17:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:17:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:17:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:17:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:17:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:17:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:17:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:17:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:17:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:17:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:17:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:17:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:17:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:17:57 oleg133-client kernel: Aug 15 07:17:57 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:17:57 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:17:57 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:17:57 oleg133-client kernel: RAX: 0000000000006796 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:17:57 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:17:57 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:17:57 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:17:57 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:17:57 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:17:57 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:17:57 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:17:57 oleg133-client kernel: Call Trace: Aug 15 07:17:57 oleg133-client kernel: Aug 15 07:17:57 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:17:57 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:17:57 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:17:57 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:17:57 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:17:57 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:17:57 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:17:57 oleg133-client kernel: Aug 15 07:17:57 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:17:57 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:17:57 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:17:57 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:17:57 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:17:57 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:17:57 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:17:57 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:17:57 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:17:57 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:17:57 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:18:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:18:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:18:24 oleg133-client kernel: Modules linked in: lustre(O) Aug 15 07:18:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:18:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:18:24 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:18:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:18:24 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:18:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:18:24 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:18:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:18:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:18:24 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:18:24 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:18:24 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:18:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:18:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:18:24 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:18:24 oleg133-client kernel: Call Trace: Aug 15 07:18:24 oleg133-client kernel: Aug 15 07:18:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:18:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:18:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:18:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:18:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:18:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:18:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:18:24 oleg133-client kernel: Aug 15 07:18:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:18:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:18:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:18:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:18:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:18:24 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:18:24 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:18:24 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:18:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:18:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:18:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:18:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:18:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:18:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:18:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:18:24 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:18:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:18:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:18:24 oleg133-client kernel: RDX: 00000000000040a8 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:18:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:18:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:18:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:18:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:18:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:18:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:18:24 oleg133-client kernel: Call Trace: Aug 15 07:18:24 oleg133-client kernel: Aug 15 07:18:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:18:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:18:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:18:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:18:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:18:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:18:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:18:24 oleg133-client kernel: Aug 15 07:18:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:18:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:18:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:18:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:18:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: osp(O) Aug 15 07:18:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:18:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:18:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:18:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:18:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:18:24 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:18:24 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:18:24 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:18:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:18:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:18:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:18:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:18:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:18:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:18:24 oleg133-client kernel: Call Trace: Aug 15 07:18:24 oleg133-client kernel: Aug 15 07:18:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:18:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:18:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:18:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:18:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:18:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:18:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:18:24 oleg133-client kernel: Aug 15 07:18:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:18:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:18:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:18:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:18:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:18:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:18:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:18:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:18:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:18:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:18:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:18:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:18:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:18:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:18:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:18:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:18:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:18:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:18:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:18:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:18:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:18:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:18:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:18:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:18:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:18:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:18:25 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:18:25 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:18:25 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:18:25 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:18:25 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:18:25 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:18:25 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:18:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:18:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:18:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:18:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:18:52 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:18:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:18:52 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:18:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix Aug 15 07:18:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:18:52 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:18:52 oleg133-client kernel: RAX: 0000000000005137 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:18:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:18:52 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:18:52 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:18:52 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:18:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:18:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:18:52 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:18:52 oleg133-client kernel: Call Trace: Aug 15 07:18:52 oleg133-client kernel: Aug 15 07:18:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:18:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:18:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:18:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:18:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:18:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:18:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:18:52 oleg133-client kernel: Aug 15 07:18:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:18:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:18:52 oleg133-client kernel: serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:18:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:18:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:18:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:18:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:18:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:18:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:18:52 oleg133-client kernel: RDX: 0000000000005e94 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:18:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:18:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:18:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:18:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:18:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:18:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:18:52 oleg133-client kernel: Call Trace: Aug 15 07:18:52 oleg133-client kernel: Aug 15 07:18:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:18:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:18:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:18:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:18:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:18:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:18:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:18:52 oleg133-client kernel: Aug 15 07:18:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:18:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:18:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:18:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:18:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:18:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:18:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:18:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:18:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:18:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:18:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:18:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:18:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:18:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:18:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:18:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:18:52 oleg133-client kernel: RAX: 0000000000003a5b RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:18:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:18:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:18:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:18:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:18:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:18:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:18:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:18:52 oleg133-client kernel: Call Trace: Aug 15 07:18:52 oleg133-client kernel: Aug 15 07:18:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:18:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:18:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:18:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:18:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:18:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:18:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:18:52 oleg133-client kernel: Aug 15 07:18:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:18:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:18:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:18:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:18:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:18:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:18:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:18:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:18:52 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:18:52 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:18:52 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:18:52 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:18:52 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:18:52 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:18:52 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:18:52 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:18:52 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:18:52 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:18:52 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:18:52 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:18:52 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:18:52 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:18:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:18:52 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:18:52 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:18:52 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:18:53 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:18:53 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:18:53 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:18:53 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:18:53 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:18:53 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:19:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:19:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:19:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:19:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:19:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:19:20 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:19:20 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:19:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:19:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:19:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:19:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:19:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:19:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:19:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:19:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:19:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:19:20 oleg133-client kernel: Call Trace: Aug 15 07:19:20 oleg133-client kernel: Aug 15 07:19:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:19:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:19:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:19:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:19:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:19:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:19:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:19:20 oleg133-client kernel: Aug 15 07:19:20 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:19:20 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:19:20 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:19:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:19:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:19:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:19:20 oleg133-client kernel: Modules linked in: Aug 15 07:19:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:19:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:19:20 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:19:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:19:20 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:19:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:19:20 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:19:20 oleg133-client kernel: RAX: 0000000000004988 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:19:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:19:20 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:19:20 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:19:20 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:19:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:19:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:19:20 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:19:20 oleg133-client kernel: Call Trace: Aug 15 07:19:20 oleg133-client kernel: Aug 15 07:19:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:19:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:19:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:19:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:19:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:19:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:19:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:19:20 oleg133-client kernel: Aug 15 07:19:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:19:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:19:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:19:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:19:20 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:19:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:19:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:19:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:19:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:19:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:19:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:19:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:19:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:19:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:19:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:19:20 oleg133-client kernel: RAX: 0000000000006f86 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:19:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:19:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:19:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:19:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:19:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:19:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:19:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:19:20 oleg133-client kernel: Call Trace: Aug 15 07:19:20 oleg133-client kernel: Aug 15 07:19:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:19:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:19:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:19:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:19:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:19:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:19:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:19:20 oleg133-client kernel: Aug 15 07:19:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:19:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:19:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:19:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:19:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:19:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:19:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:19:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:19:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:19:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:19:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:19:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:19:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:19:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:19:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:19:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:19:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:19:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:19:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:19:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:19:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:19:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:19:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:19:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:19:21 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:19:21 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:19:21 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:19:21 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:19:21 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:19:21 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:19:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 07:19:34 oleg133-client kernel: rcu: #0111-....: (2099607 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=602716 Aug 15 07:19:34 oleg133-client kernel: #011(t=3120076 jiffies g=278549 q=16094) Aug 15 07:19:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 07:19:34 oleg133-client kernel: NMI backtrace for cpu 0 Aug 15 07:19:34 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:19:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:19:34 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:19:34 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:19:34 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 Aug 15 07:19:34 oleg133-client kernel: RAX: 000000000000634f RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:19:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:19:34 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:19:34 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:19:34 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:19:34 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:19:34 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:19:34 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:19:34 oleg133-client kernel: Call Trace: Aug 15 07:19:34 oleg133-client kernel: Aug 15 07:19:34 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:19:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x98/0xa2 Aug 15 07:19:34 oleg133-client kernel: ? nmi_cpu_backtrace_handler+0x15/0x40 Aug 15 07:19:34 oleg133-client kernel: ? nmi_handle+0xbd/0x210 Aug 15 07:19:34 oleg133-client kernel: ? default_do_nmi+0x4b/0x1b0 Aug 15 07:19:34 oleg133-client kernel: ? do_nmi+0x280/0x380 Aug 15 07:19:34 oleg133-client kernel: ? end_repeat_nmi+0x16/0x1f Aug 15 07:19:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:19:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:19:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:19:34 oleg133-client kernel: Aug 15 07:19:34 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:19:34 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:19:34 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:19:34 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:19:34 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:19:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:19:34 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:19:34 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:19:34 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:19:34 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:19:34 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:19:34 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:19:34 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:19:34 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:19:34 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:19:34 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:19:34 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:19:34 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:19:34 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:19:34 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:19:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:19:34 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:19:34 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:19:34 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:19:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 07:19:34 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ] Aug 15 07:19:35 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:19:35 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:19:35 oleg133-client kernel: Call Trace: Aug 15 07:19:35 oleg133-client kernel: Aug 15 07:19:35 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 07:19:35 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 07:19:35 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 07:19:35 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 07:19:35 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 07:19:35 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 07:19:35 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 07:19:35 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 07:19:35 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 07:19:35 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 07:19:35 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 07:19:35 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:19:35 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 07:19:35 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:19:35 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:19:35 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:19:35 oleg133-client kernel: Aug 15 07:19:35 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:19:35 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:19:35 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 07:19:35 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 07:19:35 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:19:35 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:19:35 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:19:35 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:19:35 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:19:35 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:19:35 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:19:35 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:19:35 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:19:35 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:19:35 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:19:35 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 07:19:35 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:19:35 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 07:19:48 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:19:48 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:19:48 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:19:48 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:19:48 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:19:48 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:19:48 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:19:48 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:19:48 oleg133-client kernel: RAX: 0000000000000c27 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:19:48 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:19:48 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:19:48 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:19:48 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:19:48 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:19:48 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:19:48 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:19:48 oleg133-client kernel: Call Trace: Aug 15 07:19:48 oleg133-client kernel: Aug 15 07:19:48 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:19:48 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:19:48 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:19:48 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:19:48 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:19:48 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:19:48 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:19:48 oleg133-client kernel: Aug 15 07:19:48 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:19:48 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:19:48 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:19:48 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:19:48 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:19:48 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:19:48 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:19:48 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:19:48 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:19:48 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:19:48 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:20:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:20:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:20:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:20:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:20:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:20:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:20:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:20:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:20:00 oleg133-client kernel: RAX: 0000000000000111 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:20:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:20:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:20:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:20:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:20:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:20:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:20:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:20:00 oleg133-client kernel: Call Trace: Aug 15 07:20:00 oleg133-client kernel: Aug 15 07:20:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:20:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:20:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:20:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:20:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:20:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:20:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:20:00 oleg133-client kernel: Aug 15 07:20:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:20:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:20:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:20:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:20:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:20:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:20:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:20:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:20:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:20:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:20:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:20:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:20:00 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:20:00 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:20:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:20:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:20:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:20:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:20:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:20:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:20:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:20:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:20:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:20:00 oleg133-client kernel: Call Trace: Aug 15 07:20:00 oleg133-client kernel: Aug 15 07:20:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:20:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:20:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:20:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:20:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:20:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:20:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:20:00 oleg133-client kernel: Aug 15 07:20:00 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:20:00 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:20:00 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:20:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:20:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:20:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:20:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:20:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:20:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:20:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:20:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:20:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:20:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:20:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:20:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:20:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:20:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:20:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:20:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:20:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:20:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:20:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:20:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:20:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:20:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:20:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:20:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:20:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:20:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:20:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:20:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:20:01 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:20:01 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:20:01 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:20:01 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:20:01 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:20:16 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:20:16 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:20:16 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:20:16 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:20:16 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:20:16 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:20:16 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:20:16 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:20:16 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:20:16 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:20:16 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:20:16 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:20:16 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:20:16 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:20:16 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:20:16 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:20:16 oleg133-client kernel: Call Trace: Aug 15 07:20:16 oleg133-client kernel: Aug 15 07:20:16 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:20:16 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:20:16 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:20:16 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:20:16 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:20:16 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:20:16 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:20:16 oleg133-client kernel: Aug 15 07:20:16 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:20:16 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:20:16 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:20:16 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:20:16 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:20:16 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:20:16 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:20:16 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:20:16 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:20:16 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:20:16 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:20:16 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:20:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:20:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:20:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:20:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:20:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:20:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:20:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:20:28 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:20:28 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:20:28 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:20:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:20:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:20:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:20:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:20:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:20:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:20:28 oleg133-client kernel: Call Trace: Aug 15 07:20:28 oleg133-client kernel: Aug 15 07:20:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:20:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:20:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:20:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:20:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:20:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:20:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:20:28 oleg133-client kernel: Aug 15 07:20:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:20:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:20:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:20:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:20:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:20:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:20:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:20:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:20:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:20:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:20:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:20:28 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:20:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:20:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:20:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:20:28 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:20:28 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:20:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:20:28 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:20:28 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:20:28 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:20:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:20:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:20:28 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:20:28 oleg133-client kernel: Call Trace: Aug 15 07:20:28 oleg133-client kernel: Aug 15 07:20:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:20:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:20:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:20:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:20:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:20:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:20:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:20:28 oleg133-client kernel: Aug 15 07:20:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:20:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:20:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:20:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:20:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:20:28 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:20:28 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:20:28 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:20:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:20:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:20:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:20:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:20:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:20:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:20:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:20:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:20:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:20:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:20:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:20:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:20:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:20:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:20:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:20:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:20:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:20:29 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:20:29 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:20:29 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:20:29 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:20:29 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:20:29 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:20:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:20:29 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:20:29 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:20:29 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:20:29 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:20:44 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:20:44 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:20:44 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:20:44 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:20:44 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:20:44 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:20:44 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:20:44 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:20:44 oleg133-client kernel: RAX: 0000000000007f86 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:20:44 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:20:44 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:20:44 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:20:44 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:20:44 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:20:44 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:20:44 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:20:44 oleg133-client kernel: Call Trace: Aug 15 07:20:44 oleg133-client kernel: Aug 15 07:20:44 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:20:44 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:20:44 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:20:44 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:20:44 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:20:44 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:20:44 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:20:44 oleg133-client kernel: Aug 15 07:20:44 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:20:44 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:20:44 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:20:44 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:20:44 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:20:44 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:20:44 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:20:44 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:20:44 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:20:44 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:20:44 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:20:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:20:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:20:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:20:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:20:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:20:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:20:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:20:56 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:20:56 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:20:56 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:20:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:20:56 oleg133-client kernel: Modules linked in: Aug 15 07:20:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:20:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:20:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:20:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:20:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:20:56 oleg133-client kernel: Call Trace: Aug 15 07:20:56 oleg133-client kernel: Aug 15 07:20:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:20:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:20:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:20:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:20:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:20:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:20:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:20:56 oleg133-client kernel: Aug 15 07:20:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:20:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:20:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:20:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:20:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:20:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:20:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:20:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:20:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:20:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:20:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:20:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:20:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:20:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:20:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:20:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:20:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:20:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:20:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:20:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:20:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:20:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:20:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:20:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:20:56 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:20:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:20:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:20:56 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:20:56 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:20:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:20:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:20:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:20:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:20:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:20:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:20:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:20:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:20:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:20:56 oleg133-client kernel: Call Trace: Aug 15 07:20:56 oleg133-client kernel: Aug 15 07:20:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:20:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:20:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:20:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:20:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:20:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:20:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:20:56 oleg133-client kernel: Aug 15 07:20:56 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:20:56 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:20:56 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:20:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:20:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:20:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:20:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:20:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:20:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:20:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:20:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:20:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:21:12 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:21:12 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:21:12 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:21:12 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:21:12 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:21:12 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:21:12 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:21:12 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:21:12 oleg133-client kernel: RAX: 0000000000007c9e RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:21:12 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:21:12 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:21:12 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:21:12 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:21:12 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:21:12 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:21:12 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:21:12 oleg133-client kernel: Call Trace: Aug 15 07:21:12 oleg133-client kernel: Aug 15 07:21:12 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:21:12 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:21:12 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:21:12 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:21:12 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:21:12 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:21:12 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:21:12 oleg133-client kernel: Aug 15 07:21:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:21:12 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:21:12 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:21:12 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:21:12 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:21:12 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:21:12 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:21:12 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:21:12 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:21:12 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:21:12 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:21:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:21:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:21:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:21:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:21:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:21:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:21:24 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:21:24 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:21:24 oleg133-client kernel: RAX: 0000000000001d13 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:21:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:21:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:21:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:21:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:21:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:21:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:21:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:21:24 oleg133-client kernel: Call Trace: Aug 15 07:21:24 oleg133-client kernel: Aug 15 07:21:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:21:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:21:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:21:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:21:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:21:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:21:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:21:24 oleg133-client kernel: Aug 15 07:21:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:21:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:21:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:21:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:21:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:21:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:21:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:21:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:21:24 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:21:24 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:21:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:21:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:21:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:21:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:21:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:21:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:21:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:21:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:21:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:21:24 oleg133-client kernel: Call Trace: Aug 15 07:21:24 oleg133-client kernel: Aug 15 07:21:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:21:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:21:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:21:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:21:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:21:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:21:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:21:24 oleg133-client kernel: Aug 15 07:21:24 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:21:24 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:21:24 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:21:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:21:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:21:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:21:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:21:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:21:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:21:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:21:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:21:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:21:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:21:24 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:21:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:21:24 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:21:24 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:21:24 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:21:24 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:21:25 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:21:25 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:21:25 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:21:25 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:21:25 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:21:25 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:21:25 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:21:25 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:21:25 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:21:25 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:21:25 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:21:25 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:21:25 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:21:25 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:21:25 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:21:25 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:21:25 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:21:25 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:21:25 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:21:25 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:21:25 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:21:40 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:21:40 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:21:40 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:21:40 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:21:40 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:21:40 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:21:40 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:21:40 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:21:40 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:21:40 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:21:40 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:21:40 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:21:40 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:21:40 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:21:40 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:21:40 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:21:40 oleg133-client kernel: Call Trace: Aug 15 07:21:40 oleg133-client kernel: Aug 15 07:21:40 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:21:40 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:21:40 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:21:40 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:21:40 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:21:40 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:21:40 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:21:40 oleg133-client kernel: Aug 15 07:21:40 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:21:40 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:21:40 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:21:40 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:21:40 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:21:40 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:21:40 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:21:40 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:21:40 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:21:40 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:21:40 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:21:40 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:21:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:21:52 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:21:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:21:52 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:21:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:21:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:21:52 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:21:52 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:21:52 oleg133-client kernel: RAX: 0000000000006a16 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:21:52 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:21:52 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:21:52 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:21:52 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:21:52 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:21:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:21:52 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:21:52 oleg133-client kernel: Call Trace: Aug 15 07:21:52 oleg133-client kernel: Aug 15 07:21:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:21:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:21:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:21:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:21:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:21:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:21:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:21:52 oleg133-client kernel: Aug 15 07:21:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:21:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:21:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:21:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:21:52 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:21:52 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:21:52 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:21:52 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:21:52 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:21:52 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:21:52 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:21:52 oleg133-client kernel: RDX: 0000000000007bd1 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:21:52 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:21:52 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:21:52 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:21:52 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:21:52 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:21:52 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:21:52 oleg133-client kernel: Call Trace: Aug 15 07:21:52 oleg133-client kernel: Aug 15 07:21:52 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:21:52 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:21:52 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:21:52 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:21:52 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:21:52 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:21:52 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:21:52 oleg133-client kernel: Aug 15 07:21:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:21:52 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:21:52 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:21:52 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:21:52 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:21:52 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:21:52 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:21:52 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:21:52 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:21:52 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:21:52 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:21:52 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:21:52 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:21:53 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:21:53 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:21:53 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:21:53 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:21:53 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:21:53 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:21:53 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:21:53 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:21:53 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:21:53 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:21:53 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:21:53 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:21:53 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:21:53 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:21:53 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:21:53 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:21:53 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:22:08 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:22:08 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:22:08 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:22:08 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:22:08 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:22:08 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:22:08 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:22:08 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:22:08 oleg133-client kernel: RAX: 000000000000760a RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:22:08 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:22:08 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:22:08 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:22:08 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:22:08 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:22:08 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:22:08 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:22:08 oleg133-client kernel: Call Trace: Aug 15 07:22:08 oleg133-client kernel: Aug 15 07:22:08 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:22:08 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:22:08 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:22:08 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:22:08 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:22:08 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:22:08 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:22:08 oleg133-client kernel: Aug 15 07:22:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:22:08 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:22:08 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:22:08 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:22:08 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:22:08 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:22:08 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:22:08 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:22:08 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:22:08 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:22:08 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:22:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] Aug 15 07:22:20 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] Aug 15 07:22:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:22:20 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:22:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:22:20 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:22:20 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:22:20 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:22:20 oleg133-client kernel: RAX: 0000000000002395 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:22:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:22:20 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:22:20 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:22:20 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:22:20 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:22:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:22:20 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:22:20 oleg133-client kernel: Call Trace: Aug 15 07:22:20 oleg133-client kernel: Aug 15 07:22:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:22:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:22:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:22:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:22:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:22:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:22:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:22:20 oleg133-client kernel: Aug 15 07:22:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:22:20 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:22:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:22:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:22:20 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:22:20 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:22:20 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:22:20 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:22:20 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:22:20 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:22:20 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:22:20 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:22:20 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:22:20 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:22:20 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:22:20 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:22:20 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:22:20 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:22:20 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:22:20 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:22:20 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:22:20 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:22:20 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:22:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:22:20 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:22:20 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:22:20 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:22:20 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:22:20 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:22:20 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:22:20 oleg133-client kernel: Call Trace: Aug 15 07:22:20 oleg133-client kernel: Aug 15 07:22:20 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:22:20 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:22:20 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:22:20 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:22:20 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:22:20 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:22:20 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:22:20 oleg133-client kernel: Aug 15 07:22:20 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:22:20 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:22:20 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:22:20 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:22:20 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:22:20 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:22:20 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:22:20 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:22:20 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:22:20 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:22:20 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:22:20 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:22:20 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:22:20 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:22:20 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:22:20 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:22:20 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:22:20 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:22:20 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:22:34 oleg133-client kernel: rcu: INFO: rcu_sched self-detected stall on CPU Aug 15 07:22:34 oleg133-client kernel: rcu: #0111-....: (2208046 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=635565 Aug 15 07:22:34 oleg133-client kernel: #011(t=3300080 jiffies g=278549 q=16937) Aug 15 07:22:34 oleg133-client kernel: Sending NMI from CPU 1 to CPUs 0: Aug 15 07:22:34 oleg133-client kernel: NMI backtrace for cpu 0 Aug 15 07:22:34 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:22:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:22:34 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:22:34 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:22:34 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 Aug 15 07:22:34 oleg133-client kernel: RAX: 0000000000005a0b RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:22:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:22:34 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:22:34 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:22:34 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:22:34 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:22:34 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:22:34 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:22:34 oleg133-client kernel: Call Trace: Aug 15 07:22:34 oleg133-client kernel: Aug 15 07:22:34 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:22:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x98/0xa2 Aug 15 07:22:34 oleg133-client kernel: ? nmi_cpu_backtrace_handler+0x15/0x40 Aug 15 07:22:34 oleg133-client kernel: ? nmi_handle+0xbd/0x210 Aug 15 07:22:34 oleg133-client kernel: ? default_do_nmi+0x4b/0x1b0 Aug 15 07:22:34 oleg133-client kernel: ? do_nmi+0x280/0x380 Aug 15 07:22:34 oleg133-client kernel: ? end_repeat_nmi+0x16/0x1f Aug 15 07:22:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:22:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:22:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:22:34 oleg133-client kernel: Aug 15 07:22:34 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:22:34 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:22:34 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:22:34 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:22:34 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:22:34 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:22:34 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:22:34 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:22:34 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:22:34 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:22:34 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:22:34 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:22:34 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:22:34 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:22:34 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:22:34 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:22:34 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:22:34 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:22:34 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:22:34 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:22:34 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:22:34 oleg133-client kernel: NMI backtrace for cpu 1 Aug 15 07:22:34 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:22:34 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:22:34 oleg133-client kernel: Call Trace: Aug 15 07:22:34 oleg133-client kernel: Aug 15 07:22:34 oleg133-client kernel: ? dump_stack+0xbb/0x10e Aug 15 07:22:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x16/0x55 Aug 15 07:22:34 oleg133-client kernel: ? nmi_cpu_backtrace.cold.8+0x24/0xa2 Aug 15 07:22:34 oleg133-client kernel: ? lapic_can_unplug_cpu.cold.28+0x55/0x55 Aug 15 07:22:34 oleg133-client kernel: ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 Aug 15 07:22:34 oleg133-client kernel: ? arch_trigger_cpumask_backtrace+0x1d/0x30 Aug 15 07:22:34 oleg133-client kernel: ? rcu_dump_cpu_stacks+0x11b/0x171 Aug 15 07:22:34 oleg133-client kernel: ? rcu_sched_clock_irq.cold.51+0xdb/0x307 Aug 15 07:22:34 oleg133-client kernel: ? update_process_times+0x6d/0xc0 Aug 15 07:22:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x27/0x90 Aug 15 07:22:34 oleg133-client kernel: ? tick_sched_timer+0x53/0xb0 Aug 15 07:22:34 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:22:34 oleg133-client kernel: ? tick_sched_handle.isra.18+0x90/0x90 Aug 15 07:22:34 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:22:34 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:22:34 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:22:34 oleg133-client kernel: Aug 15 07:22:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:22:34 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:22:34 oleg133-client kernel: ? do_raw_spin_lock+0x177/0x190 Aug 15 07:22:34 oleg133-client kernel: ? _raw_spin_lock+0x21/0x30 Aug 15 07:22:34 oleg133-client kernel: ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: ? obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: ? mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:22:34 oleg133-client kernel: ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: ? after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:22:34 oleg133-client kernel: ? ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:22:34 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:22:34 oleg133-client kernel: ? kthread+0x1d1/0x200 Aug 15 07:22:34 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:22:34 oleg133-client kernel: ? ret_from_fork+0x1f/0x30 Aug 15 07:22:36 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:22:36 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:22:36 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:22:36 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:22:36 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:22:36 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:22:36 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:22:36 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:22:36 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:22:36 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:22:36 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:22:36 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:22:36 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:22:36 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:22:36 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:22:36 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:22:36 oleg133-client kernel: Call Trace: Aug 15 07:22:36 oleg133-client kernel: Aug 15 07:22:36 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:22:36 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:22:36 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:22:36 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:22:36 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:22:36 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:22:36 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:22:36 oleg133-client kernel: Aug 15 07:22:36 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:22:36 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:22:36 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:22:36 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:22:36 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:22:36 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:22:36 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:22:36 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:22:36 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:22:36 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:22:36 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:22:36 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:23:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:23:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:23:00 oleg133-client kernel: Modules linked in: Aug 15 07:23:00 oleg133-client kernel: Modules linked in: Aug 15 07:23:00 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:23:00 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:23:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:23:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:23:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:23:00 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:23:00 oleg133-client kernel: RAX: 00000000000055aa RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:23:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:23:00 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:23:00 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:23:00 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:23:00 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:23:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:23:00 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:23:00 oleg133-client kernel: Call Trace: Aug 15 07:23:00 oleg133-client kernel: Aug 15 07:23:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:23:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:23:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:23:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:23:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:23:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:23:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:23:00 oleg133-client kernel: Aug 15 07:23:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:23:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:23:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:23:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:23:00 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:23:00 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:23:00 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:23:00 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:23:00 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:23:00 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:23:00 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:23:00 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:23:00 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:23:00 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:23:00 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:23:00 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:23:00 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:23:00 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:23:00 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:23:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:23:00 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:23:00 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:23:00 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:23:00 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:23:00 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:23:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:23:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:23:00 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:23:00 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:23:00 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:23:00 oleg133-client kernel: RDX: 0000000000003229 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:23:00 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:23:00 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:23:00 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:23:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:23:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:23:00 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:23:00 oleg133-client kernel: Call Trace: Aug 15 07:23:00 oleg133-client kernel: Aug 15 07:23:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:23:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:23:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:23:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:23:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:23:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:23:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:23:00 oleg133-client kernel: Aug 15 07:23:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:23:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:23:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:23:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:23:00 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:23:00 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:23:00 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:23:00 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:23:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:23:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:23:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:23:04 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:23:04 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:23:04 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:23:04 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:23:04 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:23:04 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:23:04 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:23:04 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:23:04 oleg133-client kernel: RAX: 0000000000007742 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:23:04 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:23:04 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:23:04 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:23:04 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:23:04 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:23:04 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:23:04 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:23:04 oleg133-client kernel: Call Trace: Aug 15 07:23:04 oleg133-client kernel: Aug 15 07:23:04 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:23:04 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:23:04 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:23:04 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:23:04 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:23:04 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:23:04 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:23:04 oleg133-client kernel: Aug 15 07:23:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:23:04 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:23:04 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:23:04 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:23:04 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:23:04 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:23:04 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:23:04 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:23:04 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:23:04 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:23:04 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:23:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:23:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:23:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:23:28 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:23:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:23:28 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:23:28 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:23:28 oleg133-client kernel: RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:23:28 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:23:28 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:23:28 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:23:28 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:23:28 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:23:28 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:23:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:23:28 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:23:28 oleg133-client kernel: Call Trace: Aug 15 07:23:28 oleg133-client kernel: Aug 15 07:23:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:23:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:23:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:23:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:23:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:23:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:23:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:23:28 oleg133-client kernel: Aug 15 07:23:28 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:23:28 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:23:28 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:23:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:23:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:23:28 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:23:28 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:23:28 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:23:28 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:23:28 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:23:28 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:23:28 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:23:28 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:23:28 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:23:28 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:23:28 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:23:28 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:23:28 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:23:28 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:23:28 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:23:28 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:23:28 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:23:28 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:23:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:23:28 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:23:28 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:23:28 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:23:29 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:23:29 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:23:29 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:23:29 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:23:29 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:23:29 oleg133-client kernel: RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:23:29 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:23:29 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:23:29 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:23:29 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:23:29 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:23:29 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:23:29 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:23:29 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:23:29 oleg133-client kernel: Call Trace: Aug 15 07:23:29 oleg133-client kernel: Aug 15 07:23:29 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:23:29 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:23:29 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:23:29 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:23:29 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:23:29 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:23:29 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:23:29 oleg133-client kernel: Aug 15 07:23:29 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:23:29 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:23:29 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:23:29 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:23:29 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:23:29 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:23:29 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:23:29 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:23:29 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:23:29 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:23:29 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:23:29 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:23:29 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:23:29 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:23:29 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:23:29 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:23:29 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:23:29 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:23:29 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:23:32 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] Aug 15 07:23:32 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:23:32 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:23:32 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:23:32 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:23:32 oleg133-client kernel: RIP: 0010:native_safe_halt+0x2e/0x50 Aug 15 07:23:32 oleg133-client kernel: Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e Aug 15 07:23:32 oleg133-client kernel: RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:23:32 oleg133-client kernel: RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:23:32 oleg133-client kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:23:32 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:23:32 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:23:32 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:23:32 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:23:32 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:23:32 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:23:32 oleg133-client kernel: Call Trace: Aug 15 07:23:32 oleg133-client kernel: Aug 15 07:23:32 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:23:32 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:23:32 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:23:32 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:23:32 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:23:32 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:23:32 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:23:32 oleg133-client kernel: Aug 15 07:23:32 oleg133-client kernel: ? native_safe_halt+0x2e/0x50 Aug 15 07:23:32 oleg133-client kernel: kvm_wait+0xcd/0xe0 Aug 15 07:23:32 oleg133-client kernel: __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:23:32 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:23:32 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:23:32 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:23:32 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:23:32 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:23:32 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:23:32 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:23:32 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:23:32 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:23:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:23:56 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:23:56 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:23:56 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:23:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:23:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:23:56 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:23:56 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:23:56 oleg133-client kernel: RAX: 0000000000002d18 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:23:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:23:56 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:23:56 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:23:56 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:23:56 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:23:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:23:56 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:23:56 oleg133-client kernel: Call Trace: Aug 15 07:23:56 oleg133-client kernel: Aug 15 07:23:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:23:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:23:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:23:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:23:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:23:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:23:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:23:56 oleg133-client kernel: Aug 15 07:23:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:23:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:23:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:23:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:23:56 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:23:56 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: Modules linked in: Aug 15 07:23:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:23:56 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:23:56 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:23:56 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:23:56 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:23:56 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:23:56 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:23:56 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:23:56 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:23:56 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:23:56 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:23:56 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:23:56 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:23:56 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:23:56 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:23:56 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:23:56 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:23:56 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:23:56 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:23:56 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:23:56 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:23:56 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:23:56 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:23:56 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:23:56 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:23:56 oleg133-client kernel: RDX: 00000000000075ef RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:23:56 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:23:56 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:23:56 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:23:56 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:23:56 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:23:56 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:23:56 oleg133-client kernel: Call Trace: Aug 15 07:23:56 oleg133-client kernel: Aug 15 07:23:56 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:23:56 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:23:56 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:23:56 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:23:56 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:23:56 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:23:56 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:23:56 oleg133-client kernel: Aug 15 07:23:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:23:56 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:23:56 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:23:56 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:23:56 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:23:56 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:23:56 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:23:56 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:23:56 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:23:56 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:23:56 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:24:00 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:24:00 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:24:00 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:24:00 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:24:00 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:24:00 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:24:00 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:24:00 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:24:00 oleg133-client kernel: RAX: 0000000000000214 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:24:00 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:24:00 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:24:00 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:24:00 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:24:00 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:24:00 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:24:00 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:24:00 oleg133-client kernel: Call Trace: Aug 15 07:24:00 oleg133-client kernel: Aug 15 07:24:00 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:24:00 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:24:00 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:24:00 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:24:00 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:24:00 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:24:00 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:24:00 oleg133-client kernel: Aug 15 07:24:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:24:00 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:24:00 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:24:00 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:24:00 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:24:00 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:24:00 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:24:00 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:24:00 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:24:00 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:24:00 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:24:01 oleg133-client anacron[12030]: Job `cron.monthly' started Aug 15 07:24:01 oleg133-client anacron[12030]: Job `cron.monthly' terminated Aug 15 07:24:01 oleg133-client anacron[12030]: Normal exit (3 jobs run) Aug 15 07:24:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] Aug 15 07:24:24 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] Aug 15 07:24:24 oleg133-client kernel: Modules linked in: Aug 15 07:24:24 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:24:24 oleg133-client kernel: CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:24:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:24:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:24:24 oleg133-client kernel: Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 Aug 15 07:24:24 oleg133-client kernel: RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:24:24 oleg133-client kernel: RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 Aug 15 07:24:24 oleg133-client kernel: RDX: 0000000000007cf2 RSI: 0000000000000003 RDI: ffff99374626eae0 Aug 15 07:24:24 oleg133-client kernel: RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac Aug 15 07:24:24 oleg133-client kernel: R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 Aug 15 07:24:24 oleg133-client kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 Aug 15 07:24:24 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 Aug 15 07:24:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:24:24 oleg133-client kernel: CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 Aug 15 07:24:24 oleg133-client kernel: Call Trace: Aug 15 07:24:24 oleg133-client kernel: Aug 15 07:24:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:24:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:24:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:24:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:24:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:24:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:24:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:24:24 oleg133-client kernel: Aug 15 07:24:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 Aug 15 07:24:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 Aug 15 07:24:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:24:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:24:24 oleg133-client kernel: __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] Aug 15 07:24:24 oleg133-client kernel: obd_mod_free+0x46/0x140 [ptlrpc] Aug 15 07:24:24 oleg133-client kernel: mdc_commit_open+0x91/0xb0 [mdc] Aug 15 07:24:24 oleg133-client kernel: ptlrpc_free_request+0x4b/0xc0 [ptlrpc] Aug 15 07:24:24 oleg133-client kernel: ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] Aug 15 07:24:24 oleg133-client kernel: after_reply+0xe2c/0x1780 [ptlrpc] Aug 15 07:24:24 oleg133-client kernel: ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] Aug 15 07:24:24 oleg133-client kernel: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:24:24 oleg133-client kernel: CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:24:24 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:24:24 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:24:24 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:24:24 oleg133-client kernel: RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 Aug 15 07:24:24 oleg133-client kernel: RAX: 0000000000003fe3 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:24:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 Aug 15 07:24:24 oleg133-client kernel: RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:24:24 oleg133-client kernel: R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 Aug 15 07:24:24 oleg133-client kernel: R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 Aug 15 07:24:24 oleg133-client kernel: FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 Aug 15 07:24:24 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:24:24 oleg133-client kernel: CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 Aug 15 07:24:24 oleg133-client kernel: Call Trace: Aug 15 07:24:24 oleg133-client kernel: Aug 15 07:24:24 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:24:24 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:24:24 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:24:24 oleg133-client kernel: ? __next_timer_interrupt+0x160/0x160 Aug 15 07:24:24 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:24:24 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:24:24 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:24:24 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:24:24 oleg133-client kernel: Aug 15 07:24:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:24:24 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:24:24 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:24:24 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:24:24 oleg133-client kernel: ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] Aug 15 07:24:24 oleg133-client kernel: ? do_raw_spin_unlock+0x75/0x190 Aug 15 07:24:24 oleg133-client kernel: ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] Aug 15 07:24:24 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:24:24 oleg133-client kernel: ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] Aug 15 07:24:24 oleg133-client kernel: mdc_reint+0x2d/0x110 [mdc] Aug 15 07:24:24 oleg133-client kernel: mdc_unlink+0x658/0x870 [mdc] Aug 15 07:24:24 oleg133-client kernel: lmv_unlink+0x34d/0xbd0 [lmv] Aug 15 07:24:24 oleg133-client kernel: ll_unlink+0x24d/0x7f0 [lustre] Aug 15 07:24:24 oleg133-client kernel: vfs_unlink+0x202/0x330 Aug 15 07:24:24 oleg133-client kernel: do_unlinkat+0x2fa/0x460 Aug 15 07:24:24 oleg133-client kernel: __x64_sys_unlink+0x37/0x50 Aug 15 07:24:24 oleg133-client kernel: do_syscall_64+0xc1/0x440 Aug 15 07:24:24 oleg133-client kernel: entry_SYSCALL_64_after_hwframe+0x49/0xae Aug 15 07:24:24 oleg133-client kernel: RIP: 0033:0x7fe02c11910b Aug 15 07:24:24 oleg133-client kernel: Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 Aug 15 07:24:24 oleg133-client kernel: RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 Aug 15 07:24:24 oleg133-client kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b Aug 15 07:24:24 oleg133-client kernel: RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 Aug 15 07:24:24 oleg133-client kernel: RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 Aug 15 07:24:24 oleg133-client kernel: R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 Aug 15 07:24:24 oleg133-client kernel: R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 Aug 15 07:24:25 oleg133-client kernel: ptlrpcd+0xa82/0xe40 [ptlrpc] Aug 15 07:24:25 oleg133-client kernel: ? do_wait_intr+0xf0/0xf0 Aug 15 07:24:25 oleg133-client kernel: ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] Aug 15 07:24:25 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:24:25 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:24:25 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:24:28 oleg133-client kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] Aug 15 07:24:28 oleg133-client kernel: Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic Aug 15 07:24:28 oleg133-client kernel: CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 Aug 15 07:24:28 oleg133-client kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 Aug 15 07:24:28 oleg133-client kernel: Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] Aug 15 07:24:28 oleg133-client kernel: RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:24:28 oleg133-client kernel: Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 Aug 15 07:24:28 oleg133-client kernel: RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Aug 15 07:24:28 oleg133-client kernel: RAX: 0000000000007914 RBX: ffff99374626eae0 RCX: 0000000000000001 Aug 15 07:24:28 oleg133-client kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 Aug 15 07:24:28 oleg133-client kernel: RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 Aug 15 07:24:28 oleg133-client kernel: R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 Aug 15 07:24:28 oleg133-client kernel: R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 Aug 15 07:24:28 oleg133-client kernel: FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 Aug 15 07:24:28 oleg133-client kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 15 07:24:28 oleg133-client kernel: CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 Aug 15 07:24:28 oleg133-client kernel: Call Trace: Aug 15 07:24:28 oleg133-client kernel: Aug 15 07:24:28 oleg133-client kernel: ? show_regs.cold.9+0x22/0x2f Aug 15 07:24:28 oleg133-client kernel: ? watchdog_timer_fn.cold.6+0x62/0x10a Aug 15 07:24:28 oleg133-client kernel: ? __hrtimer_run_queues+0x1c2/0x4d0 Aug 15 07:24:28 oleg133-client kernel: ? watchdog+0x40/0x40 Aug 15 07:24:28 oleg133-client kernel: ? hrtimer_interrupt+0x128/0x2c0 Aug 15 07:24:28 oleg133-client kernel: ? smp_apic_timer_interrupt+0xa6/0x270 Aug 15 07:24:28 oleg133-client kernel: ? apic_timer_interrupt+0xf/0x20 Aug 15 07:24:28 oleg133-client kernel: Aug 15 07:24:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 Aug 15 07:24:28 oleg133-client kernel: ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 Aug 15 07:24:28 oleg133-client kernel: do_raw_spin_lock+0x177/0x190 Aug 15 07:24:28 oleg133-client kernel: _raw_spin_lock+0x21/0x30 Aug 15 07:24:28 oleg133-client kernel: ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] Aug 15 07:24:28 oleg133-client kernel: process_one_work+0x2c8/0x700 Aug 15 07:24:28 oleg133-client kernel: worker_thread+0x296/0x6e0 Aug 15 07:24:28 oleg133-client kernel: ? rescuer_thread+0x570/0x570 Aug 15 07:24:28 oleg133-client kernel: kthread+0x1d1/0x200 Aug 15 07:24:28 oleg133-client kernel: ? set_kthread_struct+0x70/0x70 Aug 15 07:24:28 oleg133-client kernel: ret_from_fork+0x1f/0x30 Aug 15 07:24:28 oleg133-client rsyslogd[1095]: imjournal: journal files changed, reloading... [v8.2102.0-15.el8_10.1 try https://www.rsyslog.com/e/0 ]