[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 505192773 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829700K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.003247] x2apic enabled [ 0.004010] Switched APIC routing to physical x2apic. [ 0.005018] kvm-guest: setup PV IPIs [ 0.008573] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.010030] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.011017] pid_max: default: 32768 minimum: 301 [ 0.012154] LSM: Security Framework initializing [ 0.013061] Yama: becoming mindful. [ 0.014042] SELinux: Initializing. [ 0.015069] *** VALIDATE selinux *** [ 0.023011] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027780] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028172] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030126] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031148] *** VALIDATE tmpfs *** [ 0.032480] *** VALIDATE proc *** [ 0.033281] *** VALIDATE cgroup *** [ 0.034013] *** VALIDATE cgroup2 *** [ 0.036237] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037205] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038014] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039033] Spectre V2 : User space: Vulnerable [ 0.040012] Speculative Store Bypass: Vulnerable [ 0.043045] debug: unmapping init [mem 0xffffffff9c259000-0xffffffff9c260fff] [ 0.046000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046768] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047028] ... version: 2 [ 0.048015] ... bit width: 48 [ 0.049018] ... generic registers: 4 [ 0.050015] ... value mask: 0000ffffffffffff [ 0.051019] ... max period: 00007fffffffffff [ 0.052016] ... fixed-purpose events: 3 [ 0.053017] ... event mask: 000000070000000f [ 0.055300] rcu: Hierarchical SRCU implementation. [ 0.057586] smp: Bringing up secondary CPUs ... [ 0.058593] x86: Booting SMP configuration: [ 0.059039] .... node #0, CPUs: #1 #2 #3 [ 0.064020] smp: Brought up 1 node, 4 CPUs [ 0.066016] smpboot: Max logical packages: 1 [ 0.067024] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.092028] node 0 deferred pages initialised in 23ms [ 0.094535] devtmpfs: initialized [ 0.095254] x86/mm: Memory block size: 128MB [ 0.098037] gcov: version magic: 0x41383552 [ 0.100294] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.101186] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.102296] pinctrl core: initialized pinctrl subsystem [ 0.103228] [ 0.103767] ************************************************************* [ 0.104018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.105018] ** ** [ 0.106017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.107030] ** ** [ 0.108015] ** This means that this kernel is built to expose internal ** [ 0.109021] ** IOMMU data structures, which may compromise security on ** [ 0.110018] ** your system. ** [ 0.111015] ** ** [ 0.112018] ** If you see this message and you are not debugging the ** [ 0.113018] ** kernel, report this immediately to your vendor! ** [ 0.114017] ** ** [ 0.115020] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.116016] ************************************************************* [ 0.117848] NET: Registered protocol family 16 [ 0.118483] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.119206] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.120070] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.121725] cpuidle: using governor menu [ 0.123894] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.126739] PCI: Using configuration type 1 for base access [ 0.129127] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.140063] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.141025] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.142423] cryptd: max_cpu_qlen set to 1000 [ 0.144265] ACPI: Added _OSI(Module Device) [ 0.145018] ACPI: Added _OSI(Processor Device) [ 0.146014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.147015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.151271] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.155590] ACPI: Interpreter enabled [ 0.156077] ACPI: PM: (supports S0 S3 S4 S5) [ 0.157015] ACPI: Using IOAPIC for interrupt routing [ 0.158146] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.159409] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.166820] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.167037] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.168017] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.169098] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.171457] acpiphp: Slot [2] registered [ 0.172232] acpiphp: Slot [5] registered [ 0.173142] acpiphp: Slot [6] registered [ 0.174090] acpiphp: Slot [3] registered [ 0.175066] acpiphp: Slot [4] registered [ 0.176057] acpiphp: Slot [7] registered [ 0.177015] acpiphp: Slot [8] registered [ 0.178038] acpiphp: Slot [9] registered [ 0.179018] acpiphp: Slot [10] registered [ 0.180080] acpiphp: Slot [11] registered [ 0.181066] acpiphp: Slot [12] registered [ 0.182058] acpiphp: Slot [13] registered [ 0.183062] acpiphp: Slot [14] registered [ 0.184057] acpiphp: Slot [15] registered [ 0.185108] acpiphp: Slot [16] registered [ 0.186083] acpiphp: Slot [17] registered [ 0.187051] acpiphp: Slot [18] registered [ 0.187985] acpiphp: Slot [19] registered [ 0.188064] acpiphp: Slot [20] registered [ 0.188958] acpiphp: Slot [21] registered [ 0.189123] acpiphp: Slot [22] registered [ 0.190019] acpiphp: Slot [23] registered [ 0.191155] acpiphp: Slot [24] registered [ 0.192109] acpiphp: Slot [25] registered [ 0.193127] acpiphp: Slot [26] registered [ 0.194115] acpiphp: Slot [27] registered [ 0.195123] acpiphp: Slot [28] registered [ 0.196130] acpiphp: Slot [29] registered [ 0.197128] acpiphp: Slot [30] registered [ 0.198129] acpiphp: Slot [31] registered [ 0.199111] PCI host bridge to bus 0000:00 [ 0.200000] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.202027] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.204027] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.207035] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.210034] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.213046] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.215256] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.218170] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.222447] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.229516] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.234039] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.237025] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.240024] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.243022] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.245475] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.247911] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.251047] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.253973] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.259015] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.270018] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.275018] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.280253] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.287025] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.292019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.306020] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.316000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.323015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.327016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.337015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.345738] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.348492] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.351430] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.354466] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.358274] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.363224] iommu: Default domain type: Passthrough [ 0.364000] SCSI subsystem initialized [ 0.364000] ACPI: bus type USB registered [ 0.366215] usbcore: registered new interface driver usbfs [ 0.368092] usbcore: registered new interface driver hub [ 0.370087] usbcore: registered new device driver usb [ 0.371134] pps_core: LinuxPPS API ver. 1 registered [ 0.373015] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.377103] PTP clock support registered [ 0.379098] EDAC MC: Ver: 3.0.0 [ 0.381553] PCI: Using ACPI for IRQ routing [ 0.383924] NetLabel: Initializing [ 0.385015] NetLabel: domain hash size = 128 [ 0.387013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.388099] NetLabel: unlabeled traffic allowed by default [ 0.390458] vgaarb: loaded [ 0.392345] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.393016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.398603] clocksource: Switched to clocksource kvm-clock [ 0.495989] VFS: Disk quotas dquot_6.6.0 [ 0.497232] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.499281] *** VALIDATE ramfs *** [ 0.500282] *** VALIDATE hugetlbfs *** [ 0.501450] pnp: PnP ACPI init [ 0.503951] pnp: PnP ACPI: found 6 devices [ 0.517579] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.519664] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.521913] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.524229] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.526319] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.528060] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.530308] NET: Registered protocol family 2 [ 0.532037] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.536221] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.538809] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.542601] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.545420] TCP: Hash tables configured (established 65536 bind 65536) [ 0.548463] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.551674] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.554260] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.556996] NET: Registered protocol family 1 [ 0.558953] RPC: Registered named UNIX socket transport module. [ 0.560527] RPC: Registered udp transport module. [ 0.561871] RPC: Registered tcp transport module. [ 0.563014] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.564898] NET: Registered protocol family 44 [ 0.566243] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.568136] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.570020] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.572138] PCI: CLS 0 bytes, default 64 [ 0.573692] Unpacking initramfs... [ 1.999570] debug: unmapping init [mem 0xffff9936fcc64000-0xffff9936fffcffff] [ 2.003647] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.005871] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.008405] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.512465] Initialise system trusted keyrings [ 2.513724] Key type blacklist registered [ 2.515172] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.522684] zbud: loaded [ 2.525195] *** VALIDATE nfs *** [ 2.526081] *** VALIDATE nfs4 *** [ 2.527261] pstore: using deflate compression [ 2.530200] Platform Keyring initialized [ 2.639853] NET: Registered protocol family 38 [ 2.641463] Key type asymmetric registered [ 2.643718] Asymmetric key parser 'x509' registered [ 2.645686] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.648572] io scheduler mq-deadline registered [ 2.650046] io scheduler kyber registered [ 2.652048] io scheduler bfq registered [ 2.654548] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.656595] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.659298] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.662180] ACPI: Power Button [PWRF] [ 2.667748] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.675690] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.685539] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.714087] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.747393] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.752751] Non-volatile memory driver v1.3 [ 2.753916] Linux agpgart interface v0.103 [ 2.781946] virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) [ 2.784955] vda: detected capacity change from 0 to 74702848 [ 2.798503] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.801491] vdb: detected capacity change from 0 to 1073741824 [ 2.807676] libphy: Fixed MDIO Bus: probed [ 2.812836] usbcore: registered new interface driver usbserial_generic [ 2.815297] usbserial: USB Serial support registered for generic [ 2.817493] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.821430] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.823209] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.825518] mousedev: PS/2 mouse device common for all mice [ 2.828347] rtc_cmos 00:05: RTC can wake from S4 [ 2.832570] rtc_cmos 00:05: registered as rtc0 [ 2.834135] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.836819] intel_pstate: CPU model not supported [ 2.838831] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.844675] hid: raw HID events driver (C) Jiri Kosina [ 2.844702] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.846653] usbcore: registered new interface driver usbhid [ 2.850593] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.851573] usbhid: USB HID core driver [ 2.851761] drop_monitor: Initializing network drop monitor service [ 2.857875] Initializing XFRM netlink socket [ 2.859535] NET: Registered protocol family 10 [ 2.861986] Segment Routing with IPv6 [ 2.863254] NET: Registered protocol family 17 [ 2.865365] mpls_gso: MPLS GSO support [ 2.870670] RAS: Correctable Errors collector initialized. [ 2.872336] AVX version of gcm_enc/dec engaged. [ 2.873673] AES CTR mode by8 optimization enabled [ 2.948838] sched_clock: Marking stable (2948816082, 0)->(3938959853, -990143771) [ 2.952402] registered taskstats version 1 [ 2.954995] Loading compiled-in X.509 certificates [ 2.957956] zswap: loaded using pool lzo/zbud [ 2.983189] Key type big_key registered [ 2.995830] Key type encrypted registered [ 2.997202] ima: No TPM chip found, activating TPM-bypass! [ 2.999142] ima: Allocated hash algorithm: sha1 [ 3.000704] ima: No architecture policies found [ 3.002359] evm: Initialising EVM extended attributes: [ 3.004035] evm: security.selinux [ 3.005276] evm: security.ima [ 3.006321] evm: security.capability [ 3.007516] evm: HMAC attrs: 0x1 [ 3.009906] rtc_cmos 00:05: setting system clock to 2026-08-15 09:51:55 UTC (1786787515) [ 3.016471] debug: unmapping init [mem 0xffffffff9d203000-0xffffffff9d3fffff] [ 3.019465] debug: unmapping init [mem 0xffffffff9bf82000-0xffffffff9c258fff] [ 3.031237] Write protecting the kernel read-only data: 28672k [ 3.034444] debug: unmapping init [mem 0xffffffff9a603000-0xffffffff9a7fffff] [ 3.037453] debug: unmapping init [mem 0xffffffff9af14000-0xffffffff9affffff] [ 3.070436] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.077369] systemd[1]: Detected virtualization kvm. [ 3.078906] systemd[1]: Detected architecture x86-64. [ 3.080761] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.108354] systemd[1]: No hostname configured. [ 3.109726] systemd[1]: Set hostname to . [ 3.111265] random: systemd: uninitialized urandom read (16 bytes read) [ 3.112985] systemd[1]: Initializing machine ID from random generator. [ 3.157071] random: ln: uninitialized urandom read (6 bytes read) [ 3.239870] random: systemd: uninitialized urandom read (16 bytes read) [ 3.242341] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.247978] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.252455] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Slices. [ OK ] Reached target Timers. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. Starting Journal Service... Starting Create Volatile Files and Directories... [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Swap. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.870239] device-mapper: uevent: version 1.0.3 [ 3.872504] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.663143] virtio_net virtio0 ens2: renamed from eth0 [ 4.712778] scsi host0: ata_piix [ 4.932318] scsi host1: ata_piix [ 4.933834] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 4.936315] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 8.857090] dracut-initqueue[591]: RTNETLINK answers: File exists [ 8.859296] random: fast init done [ 9.512777] random: crng init done [ 9.514092] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 9.801635] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Create Volatile Files and Directories. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 10.965090] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.290634] SELinux: Disabled at runtime. [ 11.359264] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.367582] systemd[1]: Detected virtualization kvm. [ 11.369196] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.928814] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.933109] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.939954] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.942933] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.945371] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.954706] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.963698] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-serial\x2dgetty.slice. Mounting POSIX Message Queue File System... Starting Remount Root and Kernel File Systems... Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-getty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on initctl Compatibility Named Pipe. [[ 12.055584] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. Mounting Huge Pages File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Reached target RPC Port Mapper. Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.551076] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.872910] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.881949] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.982225] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.996168] EDAC sbridge: Ver: 1.1.2 [ 14.024286] Key type dns_resolver registered [ 14.334633] NFS: Registering the id_resolver key type [ 14.336636] Key type id_resolver registered [ 14.338094] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg133-client login: [ 67.327178] libcfs: loading out-of-tree module taints kernel. [ 67.399927] Key type ._llcrypt registered [ 67.417478] Key type .llcrypt registered [ 68.072600] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 68.082155] alg: No test for adler32 (adler32-zlib) [ 69.430328] Lustre: Lustre: Build Version: 2.17.57_1_g05abe65 [ 70.360297] LNet: Added LNI 192.168.201.33@tcp [8/256/0/180] [ 72.087332] Key type lgssc registered [ 73.787271] Lustre: Echo OBD driver; http://www.lustre.org/ [ 205.984162] hrtimer: interrupt took 3153896 ns [ 220.974224] Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 [ 225.844623] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 239.815659] Lustre: DEBUG MARKER: oleg133-client.virtnet: executing check_logdir /tmp/testlogs/ [ 246.755679] Lustre: lustre-OST0000-osc-ffff993747ebd000: disconnect after 24s idle [ 246.826640] Lustre: DEBUG MARKER: oleg133-client.virtnet: executing yml_node [ 254.130536] Lustre: DEBUG MARKER: Client: 2.17.57.1 [ 257.275720] Lustre: DEBUG MARKER: MDS: 2.17.57.1 [ 261.104257] Lustre: DEBUG MARKER: OSS: 2.17.57.1 [ 262.926254] Lustre: DEBUG MARKER: -----============= acceptance-small: recovery-small ============----- Sat Aug 15 05:56:13 EDT 2026 [ 282.512235] Lustre: DEBUG MARKER: excepting tests: 136 [ 284.588532] Lustre: DEBUG MARKER: === recovery-small: start setup 05:56:35 (1786787795) === [ 291.382981] Lustre: DEBUG MARKER: oleg133-client.virtnet: executing check_config_client /mnt/lustre [ 308.026340] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 322.670093] Lustre: DEBUG MARKER: === recovery-small: finish setup 05:57:13 (1786787833) === [ 325.312215] Lustre: DEBUG MARKER: == recovery-small test 1: create, chmod, stat: drop req, drop rep ========================================================== 05:57:15 (1786787835) [ 342.495214] Lustre: 10039:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787838/real 1786787838] req@ffff993745396a00 x1873582575595904/t0(0) o700->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:30/10 lens 264/248 e 0 to 1 dl 1786787854 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'mcreate.0' uid:0 gid:0 projid:4294967295 [ 342.516512] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 342.569116] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 361.440524] Lustre: 10060:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787857/real 1786787857] req@ffff993760341f80 x1873582575597696/t0(0) o36->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 520/576 e 0 to 1 dl 1786787873 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'mcreate.0' uid:0 gid:0 projid:4294967295 [ 361.471126] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 361.509240] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 378.847215] Lustre: 10086:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787875/real 1786787875] req@ffff993760046680 x1873582575598976/t0(0) o101->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 576/1152 e 0 to 1 dl 1786787891 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'tchmod.0' uid:0 gid:0 projid:0 [ 378.894940] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 378.931907] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 396.767389] Lustre: 10107:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787893/real 1786787893] req@ffff9937600a9c00 x1873582575601152/t0(0) o36->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 488/512 e 0 to 1 dl 1786787909 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'tchmod.0' uid:0 gid:0 projid:4294967295 [ 396.794343] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 396.837262] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 414.689948] Lustre: 10133:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787911/real 1786787911] req@ffff993760046d80 x1873582575602432/t0(0) o34->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 472/728 e 0 to 1 dl 1786787927 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'statone.0' uid:0 gid:0 projid:0 [ 414.713579] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 414.748930] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 433.119352] Lustre: 10155:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787929/real 1786787929] req@ffff9937600a8700 x1873582575603712/t0(0) o34->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 472/728 e 0 to 1 dl 1786787945 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'statone.0' uid:0 gid:0 projid:0 [ 433.184104] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 433.264659] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 443.762993] Lustre: DEBUG MARKER: == recovery-small test 4: open: drop req, drop rep ======= 05:59:14 (1786787954) [ 460.767473] Lustre: 10760:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786787957/real 1786787957] req@ffff9937600a8700 x1873582575606272/t0(0) o101->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 576/1152 e 0 to 1 dl 1786787973 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 [ 460.818622] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 460.869823] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 487.657631] Lustre: DEBUG MARKER: == recovery-small test 5: rename: drop req, drop rep ===== 05:59:58 (1786787998) [ 505.312613] Lustre: 11382:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786788001/real 1786788001] req@ffff993760044000 x1873582575612416/t0(0) o101->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 576/1152 e 0 to 1 dl 1786788017 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'mv.0' uid:0 gid:0 projid:0 [ 505.365805] Lustre: 11382:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 505.376897] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 505.398955] Lustre: Skipped 1 previous similar message [ 505.458429] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 505.475303] Lustre: Skipped 1 previous similar message [ 531.597533] Lustre: DEBUG MARKER: == recovery-small test 6: link, unlink: drop req, drop rep ========================================================== 06:00:42 (1786788042) [ 549.346773] Lustre: lustre-OST0001-osc-ffff993747ebd000: disconnect after 21s idle [ 549.356539] Lustre: Skipped 1 previous similar message [ 589.280204] Lustre: 12094:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786788085/real 1786788085] req@ffff993760047480 x1873582575625856/t0(0) o36->lustre-MDT0000-mdc-ffff993747ebd000@192.168.201.133@tcp:12/10 lens 504/624 e 0 to 1 dl 1786788101 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'unlink.0' uid:0 gid:0 projid:4294967295 [ 589.311062] Lustre: 12094:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [ 589.343122] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 589.373102] Lustre: Skipped 3 previous similar messages [ 589.440892] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 589.453990] Lustre: Skipped 3 previous similar messages [ 600.274296] Lustre: DEBUG MARKER: == recovery-small test 8: touch: drop rep (bug 1423) ===== 06:01:50 (1786788110) [ 630.361177] Lustre: DEBUG MARKER: == recovery-small test 9: pause bulk on OST (bug 1420) === 06:02:21 (1786788141) [ 647.747715] Lustre: DEBUG MARKER: == recovery-small test 10a: finish request on server after client eviction (bug 1521) ========================================================== 06:02:38 (1786788158) [ 648.261563] Lustre: *** cfs_fail_loc=305, val=0*** [ 664.544377] Lustre: lustre-OST0001-osc-ffff993747ebd000: disconnect after 20s idle [ 664.848776] Lustre: *** cfs_fail_loc=305, val=0*** [ 680.235947] Lustre: *** cfs_fail_loc=305, val=0*** [ 696.589308] Lustre: *** cfs_fail_loc=305, val=0*** [ 712.999283] Lustre: *** cfs_fail_loc=305, val=0*** [ 729.354392] Lustre: *** cfs_fail_loc=305, val=0*** [ 745.741636] Lustre: *** cfs_fail_loc=305, val=0*** [ 761.463369] LustreError: lustre-MDT0000-mdc-ffff993747ebd000: operation ldlm_enqueue to node 192.168.201.133@tcp failed: rc = -107 [ 761.469840] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 761.478939] Lustre: Skipped 1 previous similar message [ 761.491608] LustreError: lustre-MDT0000-mdc-ffff993747ebd000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. [ 761.526167] Lustre: lustre-MDT0000-mdc-ffff993747ebd000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 761.533474] Lustre: Skipped 1 previous similar message [ 770.488341] Lustre: DEBUG MARKER: == recovery-small test 10b: re-send BL AST =============== 06:04:41 (1786788281) [ 787.423536] Lustre: lustre-OST0001-osc-ffff993747ebd000: disconnect after 24s idle [ 796.623890] Lustre: DEBUG MARKER: == recovery-small test 10c: re-send BL AST vs reconnect race (LU-5569) ========================================================== 06:05:07 (1786788307) [ 797.452648] Lustre: *** cfs_fail_loc=305, val=0*** [ 797.455799] Lustre: Skipped 1 previous similar message [ 806.304381] Lustre: DEBUG MARKER: == recovery-small test 10d: test failed blocking ast ===== 06:05:17 (1786788317) [ 809.729230] Lustre: Unmounted lustre-client [ 810.158720] Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 [ 812.095152] LustreError: lustre-OST0000-osc-ffff993744fd2000: operation ldlm_enqueue to node 192.168.201.133@tcp failed: rc = -107 [ 812.116718] LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 812.167778] Lustre: 2376:0:(llite_lib.c:4340:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.201.133@tcp:/lustre/fid: [0x200000403:0x1:0x0]/ may get corrupted (rc -108) [ 815.210546] Lustre: Unmounted lustre-client [ 822.365346] Lustre: DEBUG MARKER: == recovery-small test 10e: re-send BL AST vs reconnect race 2 ========================================================== 06:05:32 (1786788332) [ 823.754450] Lustre: DEBUG MARKER: SKIP: recovery-small test_10e need two clients [ 825.638866] Lustre: DEBUG MARKER: == recovery-small test 11: wake up a thread waiting for completion after eviction (b=2460) ========================================================== 06:05:36 (1786788336) [ 852.382983] Lustre: DEBUG MARKER: == recovery-small test 12: recover from timed out resend in ptlrpcd (b=2494) ========================================================== 06:06:03 (1786788363) [ 852.496890] Lustre: DEBUG MARKER: multiop /mnt/lustre/f12.recovery-small OS_c [ 870.367200] Lustre: 17679:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786788366/real 1786788366] req@ffff993760045880 x1873582575693824/t0(0) o35->lustre-MDT0000-mdc-ffff993744fd2000@192.168.201.133@tcp:23/10 lens 392/624 e 0 to 1 dl 1786788382 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 [ 870.408488] Lustre: 17679:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 898.492038] Lustre: DEBUG MARKER: == recovery-small test 13: mdc_readpage restart test (bug 1138) ========================================================== 06:06:48 (1786788408) [ 926.173853] Lustre: DEBUG MARKER: == recovery-small test 14: mdc_readpage resend test (bug 1138) ========================================================== 06:07:16 (1786788436) [ 935.629490] Lustre: DEBUG MARKER: == recovery-small test 15: failed open (-ENOMEM) ========= 06:07:26 (1786788446) [ 942.495609] Lustre: DEBUG MARKER: == recovery-small test 16: timeout bulk put, don't evict client (2732) ========================================================== 06:07:33 (1786788453) [ 989.654860] Lustre: DEBUG MARKER: == recovery-small test 17a: timeout bulk get, don't evict client (2732) ========================================================== 06:08:20 (1786788500) [ 1042.728873] Lustre: DEBUG MARKER: == recovery-small test 17b: timeout bulk get, dont evict client (3582) ========================================================== 06:09:14 (1786788554) [ 1044.154784] Lustre: DEBUG MARKER: SKIP: recovery-small test_17b Needs multiple clients [ 1046.100516] Lustre: DEBUG MARKER: == recovery-small test 18a: manual ost invalidate clears page cache immediately ========================================================== 06:09:16 (1786788556) [ 1046.943058] Lustre: setting import lustre-OST0001_UUID INACTIVE by administrator request [ 1046.990865] Lustre: lustre-OST0001-osc-ffff993744fd2000: Connection to lustre-OST0001 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 1046.999496] Lustre: Skipped 5 previous similar messages [ 1047.006469] LustreError: lustre-OST0001-osc-ffff993744fd2000: This client was evicted by lustre-OST0001; in progress operations using this service will fail. [ 1047.015845] Lustre: lustre-OST0001-osc-ffff993744fd2000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 1047.025542] Lustre: Skipped 4 previous similar messages [ 1054.374600] Lustre: DEBUG MARKER: == recovery-small test 18b: eviction and reconnect clears page cache (2766) ========================================================== 06:09:25 (1786788565) [ 1061.370247] LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1085.135528] Lustre: DEBUG MARKER: == recovery-small test 18c: Dropped connect reply after eviction handing (14755) ========================================================== 06:09:56 (1786788596) [ 1086.969643] LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1108.238394] Lustre: DEBUG MARKER: == recovery-small test 19a: test expired_lock_main on mds (2867) ========================================================== 06:10:19 (1786788619) [ 1108.811379] Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 [ 1108.817753] Lustre: Skipped 1 previous similar message [ 1141.728173] Lustre: 2376:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786788638/real 1786788638] req@ffff993760047480 x1873582575751424/t0(0) o103->lustre-MDT0000-mdc-ffff993744fd2000@192.168.201.133@tcp:17/18 lens 328/224 e 0 to 1 dl 1786788654 ref 1 fl Rpc:XQr/202/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 [ 1141.755478] Lustre: 2376:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [ 1212.409528] LustreError: lustre-MDT0000-mdc-ffff993744fd2000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. [ 1214.377145] Lustre: Unmounted lustre-client [ 1222.632974] Lustre: DEBUG MARKER: == recovery-small test 19b: test expired_lock_main on ost (2867) ========================================================== 06:12:14 (1786788734) [ 1223.109671] Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 [ 1327.616352] LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1330.561318] Lustre: Unmounted lustre-client [ 1340.121761] Lustre: DEBUG MARKER: == recovery-small test 19c: check reconnect and lock resend do not trigger expired_lock_main ========================================================== 06:14:10 (1786788850) [ 1341.218812] Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 [ 1347.777478] Lustre: Unmounted lustre-client [ 1361.492405] Lustre: DEBUG MARKER: == recovery-small test 20a: ldlm_handle_enqueue error (should return error) ========================================================== 06:14:32 (1786788872) [ 1362.675907] LustreError: lustre-OST0000-osc-ffff993744fd2000: operation ldlm_enqueue to node 192.168.201.133@tcp failed: rc = -12 [ 1369.494527] Lustre: DEBUG MARKER: == recovery-small test 20b: ldlm_handle_enqueue error (should return error) ========================================================== 06:14:40 (1786788880) [ 1370.882729] LustreError: lustre-OST0000-osc-ffff993744fd2000: operation ldlm_enqueue to node 192.168.201.133@tcp failed: rc = -12 [ 1377.966313] Lustre: DEBUG MARKER: == recovery-small test 21a: drop close request while close and open are both in flight ========================================================== 06:14:49 (1786788889) [ 1408.212483] Lustre: DEBUG MARKER: == recovery-small test 21b: drop open request while close and open are both in flight ========================================================== 06:15:19 (1786788919) [ 1553.865924] Lustre: DEBUG MARKER: == recovery-small test 21c: drop both request while close and open are both in flight ========================================================== 06:17:45 (1786789065) [ 1576.930810] Lustre: lustre-MDT0000-mdc-ffff993744fd2000: Connection to lustre-MDT0000 (at 192.168.201.133@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 1576.946421] Lustre: Skipped 18 previous similar messages [ 1576.972879] Lustre: lustre-MDT0000-mdc-ffff993744fd2000: Connection restored to 192.168.201.133@tcp (at 192.168.201.133@tcp) [ 1576.984418] Lustre: Skipped 16 previous similar messages [ 1586.776696] Lustre: DEBUG MARKER: == recovery-small test 21d: drop close reply while close and open are both in flight ========================================================== 06:18:17 (1786789097) [ 1618.365725] Lustre: DEBUG MARKER: == recovery-small test 21e: drop open reply while close and open are both in flight ========================================================== 06:18:48 (1786789128) [ 1756.127214] Lustre: 29683:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786789132/real 1786789132] req@ffff993760343480 x1873582575862400/t0(0) o36->lustre-MDT0000-mdc-ffff993744fd2000@192.168.201.133@tcp:12/10 lens 488/512 e 0 to 1 dl 1786789268 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:4294967295 [ 1756.164521] Lustre: 29683:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 14 previous similar messages [ 1758.384711] Lustre: DEBUG MARKER: == recovery-small test 21f: drop both reply while close and open are both in flight ========================================================== 06:21:09 (1786789269) [ 1785.721991] Lustre: DEBUG MARKER: == recovery-small test 21g: drop open reply and close request while close and open are both in flight ========================================================== 06:21:37 (1786789297) [ 1813.465861] Lustre: DEBUG MARKER: == recovery-small test 21h: drop open request and close reply while close and open are both in flight ========================================================== 06:22:04 (1786789324) [ 1838.762362] Lustre: DEBUG MARKER: == recovery-small test 22: drop close request and do mknod ========================================================== 06:22:30 (1786789350) [ 1864.215852] Lustre: DEBUG MARKER: == recovery-small test 23: client hang when close a file after mds crash ========================================================== 06:22:55 (1786789375) [ 1892.319644] LustreError: MGC192.168.201.133@tcp: Connection to MGS (at 192.168.201.133@tcp) was lost; in progress operations using this service will fail [ 1901.620146] Lustre: Evicted from MGS (at 192.168.201.133@tcp) after server handle changed from 0x708664883ac6d383 to 0x708664883ac6f82e [ 1902.925715] Lustre: 15962:0:(mgc_request.c:1899:mgc_process_log()) MGC192.168.201.133@tcp: IR log lustre-cliir failed, not fatal: rc = -2 [ 1922.601945] Lustre: DEBUG MARKER: oleg133-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1924.702335] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1933.959565] Lustre: DEBUG MARKER: == recovery-small test 24a: fsync error (should return error) ========================================================== 06:24:05 (1786789445) [ 1935.841560] LustreError: lustre-OST0000-osc-ffff993744fd2000: operation ost_write to node 192.168.201.133@tcp failed: rc = -107 [ 1935.853282] LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1935.886576] Lustre: 2376:0:(llite_lib.c:4340:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.201.133@tcp:/lustre/fid: [0x200000404:0x30:0x0]// may get corrupted (rc -5) [ 1935.918189] LustreError: 34177:0:(ldlm_resource.c:1207:ldlm_resource_complain()) lustre-OST0000-osc-ffff993744fd2000: namespace resource [0x240000400:0x42:0x0].0x0 (ffff993742dbfb00) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 1944.706860] Lustre: DEBUG MARKER: == recovery-small test 24b: test dirty page discard due to client eviction ========================================================== 06:24:15 (1786789455) [ 1947.315312] LustreError: lustre-OST0000-osc-ffff993744fd2000: operation ost_sync to node 192.168.201.133@tcp failed: rc = -107 [ 1947.344800] LustreError: lustre-OST0000-osc-ffff993744fd2000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1947.373406] Lustre: 2376:0:(llite_lib.c:4340:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.201.133@tcp:/lustre/fid: [0x200000404:0x35:0x0]// may get corrupted (rc -108) [ 1955.090521] Lustre: DEBUG MARKER: == recovery-small test 26a: evict dead exports =========== 06:24:26 (1786789466) [ 1957.581256] Lustre: DEBUG MARKER: SKIP: recovery-small test_26a msg and ost1 are at the same node [ 1959.719489] Lustre: DEBUG MARKER: == recovery-small test 26b: evict dead exports =========== 06:24:30 (1786789470) [ 1961.744796] Lustre: DEBUG MARKER: SKIP: recovery-small test_26b msg and ost1 are at the same node [ 1963.821739] Lustre: DEBUG MARKER: == recovery-small test 27: fail LOV while using OSC's ==== 06:24:34 (1786789474) [ 1968.564448] LustreError: lustre-MDT0000-mdc-ffff993744fd2000: operation mds_reint to node 192.168.201.133@tcp failed: rc = -19 [ 1968.574710] LustreError: Skipped 1 previous similar message [ 1989.920315] LustreError: MGC192.168.201.133@tcp: Connection to MGS (at 192.168.201.133@tcp) was lost; in progress operations using this service will fail [ 1989.959710] Lustre: Evicted from MGS (at 192.168.201.133@tcp) after server handle changed from 0x708664883ac6f82e to 0x708664883ac711bf [ 1991.794223] Lustre: 15962:0:(mgc_request.c:1899:mgc_process_log()) MGC192.168.201.133@tcp: IR log lustre-cliir failed, not fatal: rc = -2 [ 2102.069334] LustreError: lustre-MDT0000-mdc-ffff993744fd2000: operation mds_reint to node 192.168.201.133@tcp failed: rc = -19 [ 2123.231600] LustreError: MGC192.168.201.133@tcp: Connection to MGS (at 192.168.201.133@tcp) was lost; in progress operations using this service will fail [ 2123.262666] Lustre: Evicted from MGS (at 192.168.201.133@tcp) after server handle changed from 0x708664883ac711bf to 0x708664883ac8a9b7 [ 2124.668703] Lustre: 15962:0:(mgc_request.c:1899:mgc_process_log()) MGC192.168.201.133@tcp: IR log lustre-cliir failed, not fatal: rc = -2 [ 2141.726271] BUG: spinlock recursion on CPU#1, ptlrpcd_rcv/2372 [ 2141.741888] lock: 0xffff99374626eae0, .magic: dead4ead, .owner: ptlrpcd_rcv/2372, .owner_cpu: 1 [ 2141.755385] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 [ 2141.766331] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2141.775585] Call Trace: [ 2141.779573] ? dump_stack+0xbb/0x10e [ 2141.782831] ? spin_dump.cold.3+0x32/0x65 [ 2141.789260] ? do_raw_spin_lock+0x10f/0x190 [ 2141.792797] ? _raw_spin_lock+0x21/0x30 [ 2141.797020] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2141.810036] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2141.822866] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2141.832152] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2141.845548] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2141.855352] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2141.862140] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2141.871918] ? __next_timer_interrupt+0x160/0x160 [ 2141.882408] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2141.885616] ? do_wait_intr+0xf0/0xf0 [ 2141.886935] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2141.899432] ? kthread+0x1d1/0x200 [ 2141.900667] ? set_kthread_struct+0x70/0x70 [ 2141.914673] ? ret_from_fork+0x1f/0x30 [ 2163.168215] LustreError: MGC192.168.201.133@tcp: Connection to MGS (at 192.168.201.133@tcp) was lost; in progress operations using this service will fail [ 2168.057021] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2168.060009] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2168.064074] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2168.064087] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2168.064155] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 [ 2168.064171] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2168.064176] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2168.064468] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2168.064531] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2168.064536] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 2168.064540] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2168.064542] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2168.064544] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2168.064546] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2168.064548] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2168.064552] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2168.064554] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2168.064556] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2168.064565] Call Trace: [ 2168.064579] [ 2168.064584] ? show_regs.cold.9+0x22/0x2f [ 2168.064594] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2168.064599] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2168.064613] ? watchdog+0x40/0x40 [ 2168.064618] ? hrtimer_interrupt+0x128/0x2c0 [ 2168.064622] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2168.064628] ? apic_timer_interrupt+0xf/0x20 [ 2168.064632] [ 2168.064635] ? native_safe_halt+0x2e/0x50 [ 2168.064639] kvm_wait+0xcd/0xe0 [ 2168.064646] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2168.064653] do_raw_spin_lock+0x177/0x190 [ 2168.064657] _raw_spin_lock+0x21/0x30 [ 2168.064661] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2168.064893] process_one_work+0x2c8/0x700 [ 2168.064900] worker_thread+0x296/0x6e0 [ 2168.064904] ? rescuer_thread+0x570/0x570 [ 2168.064907] kthread+0x1d1/0x200 [ 2168.064912] ? set_kthread_struct+0x70/0x70 [ 2168.064916] ret_from_fork+0x1f/0x30 [ 2168.071612] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2168.071859] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2168.071867] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2168.071870] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2168.071887] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2168.071891] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2168.071896] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2168.071898] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2168.071900] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2168.071902] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2168.071904] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2168.071910] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2168.071913] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2168.071916] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2168.071923] Call Trace: [ 2168.071928] [ 2168.071938] ? show_regs.cold.9+0x22/0x2f [ 2168.071947] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2168.071954] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2168.071958] ? watchdog+0x40/0x40 [ 2168.071963] ? hrtimer_interrupt+0x128/0x2c0 [ 2168.071966] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2168.071972] ? apic_timer_interrupt+0xf/0x20 [ 2168.071982] [ 2168.071985] ? native_safe_halt+0x2e/0x50 [ 2168.071989] kvm_wait+0xcd/0xe0 [ 2168.071997] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2168.072004] do_raw_spin_lock+0x177/0x190 [ 2168.072008] _raw_spin_lock+0x21/0x30 [ 2168.072013] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2168.072299] ? do_raw_spin_unlock+0x75/0x190 [ 2168.072305] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2168.072509] ? do_wait_intr+0xf0/0xf0 [ 2168.072517] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2168.072730] mdc_reint+0x2d/0x110 [mdc] [ 2168.072768] mdc_unlink+0x658/0x870 [mdc] [ 2168.072793] lmv_unlink+0x34d/0xbd0 [lmv] [ 2168.072821] ll_unlink+0x24d/0x7f0 [lustre] [ 2168.072934] vfs_unlink+0x202/0x330 [ 2168.072942] do_unlinkat+0x2fa/0x460 [ 2168.072946] __x64_sys_unlink+0x37/0x50 [ 2168.072949] do_syscall_64+0xc1/0x440 [ 2168.072956] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2168.072962] RIP: 0033:0x7fe02c11910b [ 2168.072968] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2168.072972] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2168.072976] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2168.072978] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2168.072980] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2168.072982] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2168.072984] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2168.573517] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2168.622339] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2168.631862] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2168.637821] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2168.642396] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2168.658899] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2168.665348] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2168.672739] RDX: 0000000000002be3 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2168.677688] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2168.682852] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2168.689481] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2168.695351] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2168.700834] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2168.704345] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2168.711452] Call Trace: [ 2168.713362] [ 2168.714904] ? show_regs.cold.9+0x22/0x2f [ 2168.716879] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2168.720025] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2168.723974] ? watchdog+0x40/0x40 [ 2168.726458] ? hrtimer_interrupt+0x128/0x2c0 [ 2168.729584] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2168.732479] ? apic_timer_interrupt+0xf/0x20 [ 2168.735713] [ 2168.738012] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2168.742844] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2168.746810] do_raw_spin_lock+0x177/0x190 [ 2168.750128] _raw_spin_lock+0x21/0x30 [ 2168.752777] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2168.757479] obd_mod_free+0x46/0x140 [ptlrpc] [ 2168.763818] mdc_commit_open+0x91/0xb0 [mdc] [ 2168.767476] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2168.772143] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2168.775747] after_reply+0xe2c/0x1780 [ptlrpc] [ 2168.780775] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2168.792330] ? __next_timer_interrupt+0x160/0x160 [ 2168.797327] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2168.801275] ? do_wait_intr+0xf0/0xf0 [ 2168.804310] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2168.808769] kthread+0x1d1/0x200 [ 2168.810617] ? set_kthread_struct+0x70/0x70 [ 2168.813867] ret_from_fork+0x1f/0x30 [ 2196.057029] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2196.060187] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2196.060755] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2196.064044] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2196.064056] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2196.064137] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2196.064143] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2196.064149] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2196.064418] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2196.064428] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2196.064432] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2196.064437] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2196.064439] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2196.064441] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2196.064443] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2196.064445] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2196.064449] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2196.064451] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2196.064454] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2196.064463] Call Trace: [ 2196.064469] [ 2196.064475] ? show_regs.cold.9+0x22/0x2f [ 2196.064485] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2196.064491] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2196.064496] ? watchdog+0x40/0x40 [ 2196.064500] ? hrtimer_interrupt+0x128/0x2c0 [ 2196.064505] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2196.064591] ? apic_timer_interrupt+0xf/0x20 [ 2196.064599] [ 2196.064602] ? native_safe_halt+0x2e/0x50 [ 2196.064607] kvm_wait+0xcd/0xe0 [ 2196.064614] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2196.064622] do_raw_spin_lock+0x177/0x190 [ 2196.064626] _raw_spin_lock+0x21/0x30 [ 2196.064631] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2196.066093] process_one_work+0x2c8/0x700 [ 2196.066103] worker_thread+0x296/0x6e0 [ 2196.066107] ? rescuer_thread+0x570/0x570 [ 2196.066111] kthread+0x1d1/0x200 [ 2196.066125] ? set_kthread_struct+0x70/0x70 [ 2196.066129] ret_from_fork+0x1f/0x30 [ 2196.083506] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2196.083520] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2196.083522] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2196.083543] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2196.083549] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2196.083554] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2196.083557] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2196.083559] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2196.083561] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2196.083564] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2196.083567] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2196.083570] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2196.083573] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2196.083580] Call Trace: [ 2196.083586] [ 2196.083591] ? show_regs.cold.9+0x22/0x2f [ 2196.083600] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2196.083607] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2196.083611] ? watchdog+0x40/0x40 [ 2196.083615] ? hrtimer_interrupt+0x128/0x2c0 [ 2196.083619] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2196.083626] ? apic_timer_interrupt+0xf/0x20 [ 2196.083631] [ 2196.083634] ? native_safe_halt+0x2e/0x50 [ 2196.083637] kvm_wait+0xcd/0xe0 [ 2196.083645] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2196.083652] do_raw_spin_lock+0x177/0x190 [ 2196.083657] _raw_spin_lock+0x21/0x30 [ 2196.083664] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2196.083938] ? do_raw_spin_unlock+0x75/0x190 [ 2196.083944] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2196.084152] ? do_wait_intr+0xf0/0xf0 [ 2196.084159] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2196.084639] mdc_reint+0x2d/0x110 [mdc] [ 2196.087625] Modules linked in: lustre(O) [ 2196.094611] mdc_unlink+0x658/0x870 [mdc] [ 2196.094659] lmv_unlink+0x34d/0xbd0 [lmv] [ 2196.098553] ll_unlink+0x24d/0x7f0 [lustre] [ 2196.098672] vfs_unlink+0x202/0x330 [ 2196.098681] do_unlinkat+0x2fa/0x460 [ 2196.098979] __x64_sys_unlink+0x37/0x50 [ 2196.098987] do_syscall_64+0xc1/0x440 [ 2196.098994] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2196.099002] RIP: 0033:0x7fe02c11910b [ 2196.099010] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2196.099014] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2196.099020] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2196.099024] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2196.099026] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2196.099029] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2196.099031] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2196.551144] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2196.567673] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2196.571909] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2196.578119] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2196.588481] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2196.600164] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2196.603212] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2196.605673] RDX: 000000000000451d RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2196.608025] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2196.610858] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2196.613338] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2196.619740] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2196.626793] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2196.629431] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2196.632441] Call Trace: [ 2196.633818] [ 2196.634668] ? show_regs.cold.9+0x22/0x2f [ 2196.636267] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2196.647675] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2196.651142] ? watchdog+0x40/0x40 [ 2196.655170] ? hrtimer_interrupt+0x128/0x2c0 [ 2196.656345] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2196.657295] ? apic_timer_interrupt+0xf/0x20 [ 2196.662024] [ 2196.663023] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2196.666079] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2196.668491] do_raw_spin_lock+0x177/0x190 [ 2196.670391] _raw_spin_lock+0x21/0x30 [ 2196.672275] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2196.674991] obd_mod_free+0x46/0x140 [ptlrpc] [ 2196.687984] mdc_commit_open+0x91/0xb0 [mdc] [ 2196.702038] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2196.710493] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2196.720951] after_reply+0xe2c/0x1780 [ptlrpc] [ 2196.722910] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2196.728130] ? __next_timer_interrupt+0x160/0x160 [ 2196.737917] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2196.741994] ? do_wait_intr+0xf0/0xf0 [ 2196.745013] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2196.748177] kthread+0x1d1/0x200 [ 2196.749848] ? set_kthread_struct+0x70/0x70 [ 2196.751384] ret_from_fork+0x1f/0x30 [ 2202.097048] rcu: INFO: rcu_sched self-detected stall on CPU [ 2202.103423] rcu: 1-....: (26356 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=8957 [ 2202.106260] (t=60005 jiffies g=278549 q=313) [ 2202.107965] Sending NMI from CPU 1 to CPUs 0: [ 2202.113795] NMI backtrace for cpu 0 [ 2202.113801] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2202.113803] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2202.113805] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2202.113806] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2202.113809] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 [ 2202.113811] RAX: 0000000000000291 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2202.113813] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2202.113814] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2202.113816] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2202.113817] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2202.113818] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2202.113820] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2202.113821] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2202.113822] Call Trace: [ 2202.113823] [ 2202.113824] ? show_regs.cold.9+0x22/0x2f [ 2202.113825] ? nmi_cpu_backtrace.cold.8+0x98/0xa2 [ 2202.113826] ? nmi_cpu_backtrace_handler+0x15/0x40 [ 2202.113827] ? nmi_handle+0xbd/0x210 [ 2202.113828] ? default_do_nmi+0x4b/0x1b0 [ 2202.113829] ? do_nmi+0x280/0x380 [ 2202.113830] ? end_repeat_nmi+0x16/0x1f [ 2202.113831] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2202.113832] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2202.113833] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2202.113834] [ 2202.113835] do_raw_spin_lock+0x177/0x190 [ 2202.113836] _raw_spin_lock+0x21/0x30 [ 2202.113837] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2202.113838] ? do_raw_spin_unlock+0x75/0x190 [ 2202.113839] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2202.113839] ? do_wait_intr+0xf0/0xf0 [ 2202.113840] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2202.113841] mdc_reint+0x2d/0x110 [mdc] [ 2202.113842] mdc_unlink+0x658/0x870 [mdc] [ 2202.113843] lmv_unlink+0x34d/0xbd0 [lmv] [ 2202.113844] ll_unlink+0x24d/0x7f0 [lustre] [ 2202.113845] vfs_unlink+0x202/0x330 [ 2202.113846] do_unlinkat+0x2fa/0x460 [ 2202.113847] __x64_sys_unlink+0x37/0x50 [ 2202.113848] do_syscall_64+0xc1/0x440 [ 2202.113849] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2202.113850] RIP: 0033:0x7fe02c11910b [ 2202.113851] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2202.113854] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2202.113857] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2202.113858] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2202.113859] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2202.113861] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2202.113862] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2202.114134] NMI backtrace for cpu 1 [ 2202.246769] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2202.251463] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2202.261765] Call Trace: [ 2202.262568] [ 2202.263814] ? dump_stack+0xbb/0x10e [ 2202.264980] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 2202.267526] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 2202.268541] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 2202.272670] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 2202.276029] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 2202.284612] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 2202.285785] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 2202.286793] ? update_process_times+0x6d/0xc0 [ 2202.289848] ? tick_sched_handle.isra.18+0x27/0x90 [ 2202.291808] ? tick_sched_timer+0x53/0xb0 [ 2202.295258] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2202.298616] ? tick_sched_handle.isra.18+0x90/0x90 [ 2202.300261] ? hrtimer_interrupt+0x128/0x2c0 [ 2202.302553] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2202.304675] ? apic_timer_interrupt+0xf/0x20 [ 2202.308447] [ 2202.309086] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2202.311818] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2202.313543] ? do_raw_spin_lock+0x177/0x190 [ 2202.315860] ? _raw_spin_lock+0x21/0x30 [ 2202.317025] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2202.320240] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2202.323573] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2202.327484] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2202.329760] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2202.332465] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2202.334384] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2202.336772] ? __next_timer_interrupt+0x160/0x160 [ 2202.338260] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2202.339726] ? do_wait_intr+0xf0/0xf0 [ 2202.340759] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2202.342644] ? kthread+0x1d1/0x200 [ 2202.343893] ? set_kthread_struct+0x70/0x70 [ 2202.345355] ? ret_from_fork+0x1f/0x30 [ 2224.064025] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2224.067498] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2224.090437] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2224.095907] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2224.100421] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2224.104101] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2224.106280] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2224.115757] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 2224.119263] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2224.122949] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2224.126345] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2224.129888] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2224.132868] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2224.136323] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2224.139822] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2224.141935] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2224.145122] Call Trace: [ 2224.146261] [ 2224.147639] ? show_regs.cold.9+0x22/0x2f [ 2224.148918] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2224.151078] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2224.152943] ? watchdog+0x40/0x40 [ 2224.154370] ? hrtimer_interrupt+0x128/0x2c0 [ 2224.155936] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2224.158276] ? apic_timer_interrupt+0xf/0x20 [ 2224.160407] [ 2224.161692] ? native_safe_halt+0x2e/0x50 [ 2224.163658] kvm_wait+0xcd/0xe0 [ 2224.165141] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2224.168078] do_raw_spin_lock+0x177/0x190 [ 2224.169936] _raw_spin_lock+0x21/0x30 [ 2224.171764] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2224.174612] process_one_work+0x2c8/0x700 [ 2224.176950] worker_thread+0x296/0x6e0 [ 2224.178916] ? rescuer_thread+0x570/0x570 [ 2224.180772] kthread+0x1d1/0x200 [ 2224.182371] ? set_kthread_struct+0x70/0x70 [ 2224.184228] ret_from_fork+0x1f/0x30 [ 2228.057070] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2228.060161] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2228.060479] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2228.060587] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2228.060593] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2228.060596] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2228.060614] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2228.060619] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2228.060624] RAX: 0000000000006474 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2228.060626] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2228.060628] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2228.060630] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2228.060632] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2228.060635] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2228.060638] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2228.060640] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2228.060648] Call Trace: [ 2228.060653] [ 2228.060659] ? show_regs.cold.9+0x22/0x2f [ 2228.060669] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2228.060682] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2228.060687] ? watchdog+0x40/0x40 [ 2228.060692] ? hrtimer_interrupt+0x128/0x2c0 [ 2228.060696] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2228.060709] ? apic_timer_interrupt+0xf/0x20 [ 2228.060715] [ 2228.060718] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2228.060723] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2228.060729] do_raw_spin_lock+0x177/0x190 [ 2228.060734] _raw_spin_lock+0x21/0x30 [ 2228.060740] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2228.065940] Modules linked in: [ 2228.088468] ? do_raw_spin_unlock+0x75/0x190 [ 2228.088487] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2228.094954] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) [ 2228.099947] ? do_wait_intr+0xf0/0xf0 [ 2228.104439] fid(O) [ 2228.113706] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2228.114212] mdc_reint+0x2d/0x110 [mdc] [ 2228.114262] mdc_unlink+0x658/0x870 [mdc] [ 2228.114287] lmv_unlink+0x34d/0xbd0 [lmv] [ 2228.114316] ll_unlink+0x24d/0x7f0 [lustre] [ 2228.119040] fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2228.119098] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2228.119108] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2228.119111] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2228.119125] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2228.119130] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2228.119135] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2228.119137] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2228.119139] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2228.119141] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2228.123432] vfs_unlink+0x202/0x330 [ 2228.123456] do_unlinkat+0x2fa/0x460 [ 2228.123462] __x64_sys_unlink+0x37/0x50 [ 2228.123466] do_syscall_64+0xc1/0x440 [ 2228.123473] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2228.123485] RIP: 0033:0x7fe02c11910b [ 2228.123494] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2228.123498] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2228.123504] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2228.123506] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2228.123512] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2228.123514] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2228.123516] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2228.322488] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2228.325676] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2228.329490] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2228.331770] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2228.334618] Call Trace: [ 2228.335493] [ 2228.336352] ? show_regs.cold.9+0x22/0x2f [ 2228.337759] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2228.339861] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2228.341653] ? watchdog+0x40/0x40 [ 2228.342707] ? hrtimer_interrupt+0x128/0x2c0 [ 2228.343967] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2228.345799] ? apic_timer_interrupt+0xf/0x20 [ 2228.347291] [ 2228.348121] ? native_safe_halt+0x2e/0x50 [ 2228.349922] kvm_wait+0xcd/0xe0 [ 2228.351278] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2228.353524] do_raw_spin_lock+0x177/0x190 [ 2228.355507] _raw_spin_lock+0x21/0x30 [ 2228.357841] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2228.361111] obd_mod_free+0x46/0x140 [ptlrpc] [ 2228.364269] mdc_commit_open+0x91/0xb0 [mdc] [ 2228.366597] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2228.368879] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2228.371820] after_reply+0xe2c/0x1780 [ptlrpc] [ 2228.374516] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2228.377362] ? __next_timer_interrupt+0x160/0x160 [ 2228.379611] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2228.381734] ? do_wait_intr+0xf0/0xf0 [ 2228.383328] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2228.386411] kthread+0x1d1/0x200 [ 2228.387826] ? set_kthread_struct+0x70/0x70 [ 2228.390507] ret_from_fork+0x1f/0x30 [ 2252.064022] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2252.067673] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2252.097937] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2252.108354] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2252.114549] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2252.118207] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2252.119977] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2252.128628] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2252.131392] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2252.134701] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2252.137388] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2252.140919] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2252.143631] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2252.145616] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2252.148065] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2252.149664] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2252.151691] Call Trace: [ 2252.152366] [ 2252.152889] ? show_regs.cold.9+0x22/0x2f [ 2252.153943] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2252.155676] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2252.157163] ? watchdog+0x40/0x40 [ 2252.158358] ? hrtimer_interrupt+0x128/0x2c0 [ 2252.159703] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2252.161116] ? apic_timer_interrupt+0xf/0x20 [ 2252.162387] [ 2252.163003] ? native_safe_halt+0x2e/0x50 [ 2252.164243] kvm_wait+0xcd/0xe0 [ 2252.165067] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2252.166563] do_raw_spin_lock+0x177/0x190 [ 2252.167791] _raw_spin_lock+0x21/0x30 [ 2252.168636] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2252.170048] process_one_work+0x2c8/0x700 [ 2252.171678] worker_thread+0x296/0x6e0 [ 2252.173048] ? rescuer_thread+0x570/0x570 [ 2252.174118] kthread+0x1d1/0x200 [ 2252.174900] ? set_kthread_struct+0x70/0x70 [ 2252.175932] ret_from_fork+0x1f/0x30 [ 2256.058332] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2256.063886] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2256.066764] Modules linked in: lustre(O) osp(O) [ 2256.082955] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2256.083036] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2256.083044] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2256.083047] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2256.083072] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2256.083077] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2256.083081] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2256.083084] RDX: 0000000000006b0e RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2256.083086] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2256.083088] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2256.083090] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2256.083093] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2256.083095] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2256.083097] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2256.083106] Call Trace: [ 2256.083112] [ 2256.083119] ? show_regs.cold.9+0x22/0x2f [ 2256.083129] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2256.083136] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2256.083140] ? watchdog+0x40/0x40 [ 2256.083144] ? hrtimer_interrupt+0x128/0x2c0 [ 2256.083148] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2256.083155] ? apic_timer_interrupt+0xf/0x20 [ 2256.083160] [ 2256.083162] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2256.084804] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2256.084809] do_raw_spin_lock+0x177/0x190 [ 2256.084814] _raw_spin_lock+0x21/0x30 [ 2256.084822] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2256.088314] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2256.088397] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2256.088404] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2256.088407] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2256.088424] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2256.088429] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2256.088433] RAX: 0000000000007f89 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2256.088435] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2256.088437] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2256.088439] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2256.088442] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2256.088445] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2256.088498] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2256.088500] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2256.088507] Call Trace: [ 2256.088512] [ 2256.088517] ? show_regs.cold.9+0x22/0x2f [ 2256.150852] obd_mod_free+0x46/0x140 [ptlrpc] [ 2256.156702] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2256.156720] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2256.156724] ? watchdog+0x40/0x40 [ 2256.156728] ? hrtimer_interrupt+0x128/0x2c0 [ 2256.156731] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2256.156739] ? apic_timer_interrupt+0xf/0x20 [ 2256.156742] [ 2256.156746] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2256.156752] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2256.156756] do_raw_spin_lock+0x177/0x190 [ 2256.156761] _raw_spin_lock+0x21/0x30 [ 2256.156767] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2256.157242] ? do_raw_spin_unlock+0x75/0x190 [ 2256.157248] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2256.157579] ? do_wait_intr+0xf0/0xf0 [ 2256.157592] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2256.157862] mdc_reint+0x2d/0x110 [mdc] [ 2256.157981] mdc_unlink+0x658/0x870 [mdc] [ 2256.158100] lmv_unlink+0x34d/0xbd0 [lmv] [ 2256.158157] ll_unlink+0x24d/0x7f0 [lustre] [ 2256.172752] mdc_commit_open+0x91/0xb0 [mdc] [ 2256.172799] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2256.173057] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2256.173353] after_reply+0xe2c/0x1780 [ptlrpc] [ 2256.173926] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2256.174127] ? __next_timer_interrupt+0x160/0x160 [ 2256.174137] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2256.178744] vfs_unlink+0x202/0x330 [ 2256.178759] do_unlinkat+0x2fa/0x460 [ 2256.178764] __x64_sys_unlink+0x37/0x50 [ 2256.178767] do_syscall_64+0xc1/0x440 [ 2256.179228] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2256.179237] RIP: 0033:0x7fe02c11910b [ 2256.179246] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2256.179249] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2256.179256] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2256.179258] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2256.179260] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2256.179262] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2256.179264] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2256.618678] ? do_wait_intr+0xf0/0xf0 [ 2256.620212] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2256.627873] kthread+0x1d1/0x200 [ 2256.629760] ? set_kthread_struct+0x70/0x70 [ 2256.631610] ret_from_fork+0x1f/0x30 [ 2280.064078] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2280.068704] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2280.119785] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2280.147141] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2280.154883] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2280.163450] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2280.172084] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2280.191132] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2280.199992] RAX: 0000000000007dfd RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2280.208764] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2280.213364] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2280.218965] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2280.225660] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2280.229262] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2280.238946] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2280.245771] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2280.254426] Call Trace: [ 2280.257606] [ 2280.258144] ? show_regs.cold.9+0x22/0x2f [ 2280.259333] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2280.261883] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2280.263800] ? watchdog+0x40/0x40 [ 2280.266921] ? hrtimer_interrupt+0x128/0x2c0 [ 2280.270150] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2280.272119] ? apic_timer_interrupt+0xf/0x20 [ 2280.277108] [ 2280.279101] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2280.284633] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2280.287839] do_raw_spin_lock+0x177/0x190 [ 2280.292722] _raw_spin_lock+0x21/0x30 [ 2280.298897] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2280.310999] process_one_work+0x2c8/0x700 [ 2280.317057] worker_thread+0x296/0x6e0 [ 2280.318307] ? rescuer_thread+0x570/0x570 [ 2280.324768] kthread+0x1d1/0x200 [ 2280.327410] ? set_kthread_struct+0x70/0x70 [ 2280.334307] ret_from_fork+0x1f/0x30 [ 2284.057010] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2284.060006] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) [ 2284.060026] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2284.060030] mgs(O) [ 2284.060039] Modules linked in: [ 2284.066188] lquota(O) [ 2284.070305] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2284.070388] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2284.070395] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2284.070403] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2284.070417] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2284.070421] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2284.070425] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2284.070427] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2284.070430] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2284.070432] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2284.070434] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2284.070437] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2284.070440] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2284.070442] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2284.070450] Call Trace: [ 2284.070455] [ 2284.070461] ? show_regs.cold.9+0x22/0x2f [ 2284.070471] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2284.070479] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2284.070484] ? watchdog+0x40/0x40 [ 2284.070488] ? hrtimer_interrupt+0x128/0x2c0 [ 2284.070492] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2284.070505] ? apic_timer_interrupt+0xf/0x20 [ 2284.070511] [ 2284.070514] ? native_safe_halt+0x2e/0x50 [ 2284.070518] kvm_wait+0xcd/0xe0 [ 2284.070526] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2284.070534] do_raw_spin_lock+0x177/0x190 [ 2284.070538] _raw_spin_lock+0x21/0x30 [ 2284.070543] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2284.070850] obd_mod_free+0x46/0x140 [ptlrpc] [ 2284.071163] mdc_commit_open+0x91/0xb0 [mdc] [ 2284.071495] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2284.071743] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2284.071949] after_reply+0xe2c/0x1780 [ptlrpc] [ 2284.075836] lfsck(O) [ 2284.077417] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2284.077696] ? __next_timer_interrupt+0x160/0x160 [ 2284.077707] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2284.077910] ? do_wait_intr+0xf0/0xf0 [ 2284.077920] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2284.078122] kthread+0x1d1/0x200 [ 2284.078128] ? set_kthread_struct+0x70/0x70 [ 2284.078132] ret_from_fork+0x1f/0x30 [ 2284.356674] obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2284.377356] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2284.396323] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2284.405856] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2284.412535] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2284.433035] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2284.439815] RAX: 0000000000005757 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2284.448140] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2284.454938] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2284.461691] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2284.469494] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2284.477519] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2284.485682] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2284.490812] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2284.495834] Call Trace: [ 2284.499376] [ 2284.501434] ? show_regs.cold.9+0x22/0x2f [ 2284.504608] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2284.510108] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2284.513250] ? watchdog+0x40/0x40 [ 2284.514380] ? hrtimer_interrupt+0x128/0x2c0 [ 2284.522389] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2284.527941] ? apic_timer_interrupt+0xf/0x20 [ 2284.534013] [ 2284.535983] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2284.539683] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2284.545626] do_raw_spin_lock+0x177/0x190 [ 2284.550476] _raw_spin_lock+0x21/0x30 [ 2284.554077] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2284.560383] ? do_raw_spin_unlock+0x75/0x190 [ 2284.563131] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2284.569629] ? do_wait_intr+0xf0/0xf0 [ 2284.572109] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2284.577900] mdc_reint+0x2d/0x110 [mdc] [ 2284.580770] mdc_unlink+0x658/0x870 [mdc] [ 2284.585390] lmv_unlink+0x34d/0xbd0 [lmv] [ 2284.588050] ll_unlink+0x24d/0x7f0 [lustre] [ 2284.593822] vfs_unlink+0x202/0x330 [ 2284.596811] do_unlinkat+0x2fa/0x460 [ 2284.604360] __x64_sys_unlink+0x37/0x50 [ 2284.610362] do_syscall_64+0xc1/0x440 [ 2284.612905] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2284.620683] RIP: 0033:0x7fe02c11910b [ 2284.622878] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2284.647485] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2284.654728] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2284.658827] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2284.663837] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2284.669867] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2284.677461] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2308.064024] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2308.066875] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2308.083844] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2308.086801] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2308.089505] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2308.091497] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2308.092792] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2308.097792] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2308.099865] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2308.101806] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2308.103661] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2308.105399] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2308.107473] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2308.109388] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2308.111499] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2308.113190] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2308.115163] Call Trace: [ 2308.115889] [ 2308.116536] ? show_regs.cold.9+0x22/0x2f [ 2308.117609] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2308.119047] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2308.121913] ? watchdog+0x40/0x40 [ 2308.123520] ? hrtimer_interrupt+0x128/0x2c0 [ 2308.124923] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2308.126684] ? apic_timer_interrupt+0xf/0x20 [ 2308.128129] [ 2308.128687] ? native_safe_halt+0x2e/0x50 [ 2308.129899] kvm_wait+0xcd/0xe0 [ 2308.130928] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2308.132626] do_raw_spin_lock+0x177/0x190 [ 2308.133814] _raw_spin_lock+0x21/0x30 [ 2308.136665] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2308.138961] process_one_work+0x2c8/0x700 [ 2308.140128] worker_thread+0x296/0x6e0 [ 2308.142106] ? rescuer_thread+0x570/0x570 [ 2308.143317] kthread+0x1d1/0x200 [ 2308.144344] ? set_kthread_struct+0x70/0x70 [ 2308.145709] ret_from_fork+0x1f/0x30 [ 2312.057028] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2312.061514] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2312.062701] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2312.062777] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2312.062783] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2312.062786] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2312.062799] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2312.062803] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2312.062807] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2312.062809] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 2312.062811] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2312.062813] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2312.062816] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2312.062819] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2312.062821] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2312.062823] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2312.062832] Call Trace: [ 2312.062838] [ 2312.062843] ? show_regs.cold.9+0x22/0x2f [ 2312.062852] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2312.062858] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2312.062862] ? watchdog+0x40/0x40 [ 2312.062867] ? hrtimer_interrupt+0x128/0x2c0 [ 2312.062870] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2312.062877] ? apic_timer_interrupt+0xf/0x20 [ 2312.062881] [ 2312.062884] ? native_safe_halt+0x2e/0x50 [ 2312.062888] kvm_wait+0xcd/0xe0 [ 2312.062894] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2312.062901] do_raw_spin_lock+0x177/0x190 [ 2312.062906] _raw_spin_lock+0x21/0x30 [ 2312.062910] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2312.118099] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2312.118176] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2312.118183] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2312.118186] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2312.118200] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2312.118205] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2312.118209] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2312.118211] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2312.118213] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2312.118215] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2312.118217] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2312.118220] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2312.118222] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2312.118225] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2312.118233] Call Trace: [ 2312.118239] [ 2312.118245] ? show_regs.cold.9+0x22/0x2f [ 2312.118256] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2312.118263] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2312.118267] ? watchdog+0x40/0x40 [ 2312.118272] ? hrtimer_interrupt+0x128/0x2c0 [ 2312.118276] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2312.118282] ? apic_timer_interrupt+0xf/0x20 [ 2312.118286] [ 2312.118289] ? native_safe_halt+0x2e/0x50 [ 2312.118293] kvm_wait+0xcd/0xe0 [ 2312.118300] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2312.118308] do_raw_spin_lock+0x177/0x190 [ 2312.118313] _raw_spin_lock+0x21/0x30 [ 2312.118318] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2312.118571] obd_mod_free+0x46/0x140 [ptlrpc] [ 2312.127401] ? do_raw_spin_unlock+0x75/0x190 [ 2312.127422] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2312.132382] mdc_commit_open+0x91/0xb0 [mdc] [ 2312.132428] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2312.132942] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2312.133162] after_reply+0xe2c/0x1780 [ptlrpc] [ 2312.135422] ? do_wait_intr+0xf0/0xf0 [ 2312.142727] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2312.148060] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2312.148358] mdc_reint+0x2d/0x110 [mdc] [ 2312.148396] mdc_unlink+0x658/0x870 [mdc] [ 2312.148423] lmv_unlink+0x34d/0xbd0 [lmv] [ 2312.148455] ll_unlink+0x24d/0x7f0 [lustre] [ 2312.148568] vfs_unlink+0x202/0x330 [ 2312.148578] do_unlinkat+0x2fa/0x460 [ 2312.148583] __x64_sys_unlink+0x37/0x50 [ 2312.148587] do_syscall_64+0xc1/0x440 [ 2312.148594] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2312.148605] RIP: 0033:0x7fe02c11910b [ 2312.148612] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2312.148617] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2312.148622] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2312.148624] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2312.148626] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2312.148628] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2312.148630] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2312.666166] ? __next_timer_interrupt+0x160/0x160 [ 2312.667778] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2312.669429] ? do_wait_intr+0xf0/0xf0 [ 2312.670772] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2312.672616] kthread+0x1d1/0x200 [ 2312.673611] ? set_kthread_struct+0x70/0x70 [ 2312.675517] ret_from_fork+0x1f/0x30 [ 2335.711102] INFO: task khugepaged:48 blocked for more than 120 seconds. [ 2335.714021] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2335.718181] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2335.723029] task:khugepaged state:D stack:0 pid:48 ppid:2 flags:0x80004000 [ 2335.733045] Call Trace: [ 2335.736101] __schedule+0x351/0xcb0 [ 2335.738078] ? wait_for_completion+0xae/0x1e0 [ 2335.741895] schedule+0xc0/0x180 [ 2335.743313] schedule_timeout+0x126/0x190 [ 2335.749050] ? try_to_wake_up+0x80/0x710 [ 2335.751014] ? __prepare_to_swait+0x5b/0x90 [ 2335.754912] ? do_raw_spin_unlock+0x75/0x190 [ 2335.764219] wait_for_completion+0xf0/0x1e0 [ 2335.765723] flush_work+0x1f3/0x390 [ 2335.766943] ? flush_workqueue_prep_pwqs+0x200/0x200 [ 2335.772233] ? do_raw_spin_unlock+0x75/0x190 [ 2335.776023] ? __pagevec_release+0x50/0x50 [ 2335.778981] lru_add_drain_all+0x1d4/0x280 [ 2335.780300] khugepaged+0xa2/0x16f0 [ 2335.784389] ? woken_wake_function+0x30/0x30 [ 2335.785950] ? collapse_pte_mapped_thp+0x8a0/0x8a0 [ 2335.787857] kthread+0x1d1/0x200 [ 2335.789040] ? set_kthread_struct+0x70/0x70 [ 2335.791591] ret_from_fork+0x1f/0x30 [ 2335.793334] INFO: task writemany:36103 blocked for more than 120 seconds. [ 2335.795837] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2335.799065] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2335.802840] task:writemany state:D stack:0 pid:36103 ppid:36100 flags:0x80000004 [ 2335.808149] Call Trace: [ 2335.809088] __schedule+0x351/0xcb0 [ 2335.810936] schedule+0xc0/0x180 [ 2335.812462] schedule_preempt_disabled+0x21/0x40 [ 2335.814585] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2335.816750] ? lprocfs_counter_add+0x15b/0x210 [obdclass] [ 2335.818928] down_write+0x80/0xd0 [ 2335.820827] do_last+0x2eb/0xfc0 [ 2335.822557] ? nd_jump_root+0xe5/0x160 [ 2335.824336] ? path_init+0x437/0x520 [ 2335.826241] path_openat+0xf7/0x500 [ 2335.828300] ? do_raw_spin_unlock+0x75/0x190 [ 2335.830233] do_filp_open+0x99/0x140 [ 2335.831890] ? getname_flags+0x6e/0x330 [ 2335.834188] ? __check_object_size+0xff/0x256 [ 2335.836329] ? do_raw_spin_unlock+0x75/0x190 [ 2335.838467] ? _raw_spin_unlock+0x12/0x30 [ 2335.840299] do_sys_openat2+0x2b4/0x410 [ 2335.842286] do_sys_open+0x73/0xa0 [ 2335.843922] __x64_sys_openat+0x24/0x30 [ 2335.845934] do_syscall_64+0xc1/0x440 [ 2335.847841] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2335.851500] RIP: 0033:0x7fe02c20e332 [ 2335.853366] Code: Unable to access opcode bytes at RIP 0x7fe02c20e308. [ 2335.857385] RSP: 002b:00007fff45ac4b90 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 [ 2335.861250] RAX: ffffffffffffffda RBX: 00000000000191d8 RCX: 00007fe02c20e332 [ 2335.864984] RDX: 0000000000000042 RSI: 00007fff45ac4c60 RDI: 00000000ffffff9c [ 2335.869295] RBP: 00007fff45ac5060 R08: 0000000000000000 R09: 0000000000000000 [ 2335.875773] R10: 00000000000001b6 R11: 0000000000000246 R12: 00007fff45ac4c60 [ 2335.880544] R13: 00000000ffffffff R14: 0000000000000191 R15: 0000000000000000 [ 2335.924573] INFO: task writemany:36104 blocked for more than 120 seconds. [ 2335.935967] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2335.948992] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2335.956555] task:writemany state:D stack:0 pid:36104 ppid:36100 flags:0x80000004 [ 2335.961832] Call Trace: [ 2335.963043] __schedule+0x351/0xcb0 [ 2335.966159] schedule+0xc0/0x180 [ 2335.967944] schedule_preempt_disabled+0x21/0x40 [ 2335.973928] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2335.978399] ? lprocfs_counter_add+0x15b/0x210 [obdclass] [ 2335.987942] down_write+0x80/0xd0 [ 2335.990961] do_last+0x2eb/0xfc0 [ 2335.994725] ? nd_jump_root+0xe5/0x160 [ 2335.997175] ? path_init+0x437/0x520 [ 2335.998158] path_openat+0xf7/0x500 [ 2336.003022] ? do_raw_spin_unlock+0x75/0x190 [ 2336.005016] do_filp_open+0x99/0x140 [ 2336.007036] ? getname_flags+0x6e/0x330 [ 2336.011268] ? __check_object_size+0xff/0x256 [ 2336.012904] ? do_raw_spin_unlock+0x75/0x190 [ 2336.019358] ? _raw_spin_unlock+0x12/0x30 [ 2336.021816] do_sys_openat2+0x2b4/0x410 [ 2336.026732] do_sys_open+0x73/0xa0 [ 2336.029231] __x64_sys_openat+0x24/0x30 [ 2336.038579] do_syscall_64+0xc1/0x440 [ 2336.044621] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2336.054218] RIP: 0033:0x7fe02c20e332 [ 2336.058260] Code: Unable to access opcode bytes at RIP 0x7fe02c20e308. [ 2336.064039] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2336.065733] RSP: 002b:00007fff45ac4b90 EFLAGS: 00000246 [ 2336.070081] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) [ 2336.072167] ORIG_RAX: 0000000000000101 [ 2336.072174] RAX: ffffffffffffffda RBX: 00000000000f0290 RCX: 00007fe02c20e332 [ 2336.072176] RDX: 0000000000000042 RSI: 00007fff45ac4c60 RDI: 00000000ffffff9c [ 2336.072177] RBP: 00007fff45ac5060 R08: 0000000000000000 R09: 0000000000000000 [ 2336.072179] R10: 00000000000001b6 R11: 0000000000000246 R12: 00007fff45ac4c60 [ 2336.072181] R13: 00000000ffffffff R14: 000000000000015f R15: 0000000000000000 [ 2336.075308] INFO: task writemany:36105 blocked for more than 120 seconds. [ 2336.089850] obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2336.089909] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2336.089917] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2336.091519] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2336.091526] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2336.091529] task:writemany state:D stack:0 pid:36105 ppid:36100 flags:0x80000004 [ 2336.091537] Call Trace: [ 2336.091545] __schedule+0x351/0xcb0 [ 2336.091559] schedule+0xc0/0x180 [ 2336.091564] schedule_preempt_disabled+0x21/0x40 [ 2336.091569] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2336.091576] ? __might_sleep+0x59/0xc0 [ 2336.091582] down_write+0x80/0xd0 [ 2336.091585] do_unlinkat+0x184/0x460 [ 2336.094455] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2336.094853] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2336.094864] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2336.094868] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2336.094872] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2336.094874] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2336.094880] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2336.094882] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2336.094884] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2336.094887] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2336.094890] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2336.094892] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2336.094899] Call Trace: [ 2336.094904] [ 2336.094909] ? show_regs.cold.9+0x22/0x2f [ 2336.094925] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2336.094933] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2336.094938] ? watchdog+0x40/0x40 [ 2336.094942] ? hrtimer_interrupt+0x128/0x2c0 [ 2336.094949] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2336.094957] ? apic_timer_interrupt+0xf/0x20 [ 2336.094962] [ 2336.094964] ? native_safe_halt+0x2e/0x50 [ 2336.094968] kvm_wait+0xcd/0xe0 [ 2336.094979] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2336.094988] do_raw_spin_lock+0x177/0x190 [ 2336.094993] _raw_spin_lock+0x21/0x30 [ 2336.094998] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2336.095276] process_one_work+0x2c8/0x700 [ 2336.095283] worker_thread+0x296/0x6e0 [ 2336.095288] ? rescuer_thread+0x570/0x570 [ 2336.095291] kthread+0x1d1/0x200 [ 2336.095296] ? set_kthread_struct+0x70/0x70 [ 2336.095300] ret_from_fork+0x1f/0x30 [ 2336.336692] __x64_sys_unlink+0x37/0x50 [ 2336.341216] do_syscall_64+0xc1/0x440 [ 2336.346565] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2336.348925] RIP: 0033:0x7fe02c11910b [ 2336.353401] Code: Unable to access opcode bytes at RIP 0x7fe02c1190e1. [ 2336.357245] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2336.361481] RAX: ffffffffffffffda RBX: 000000000058d508 RCX: 00007fe02c11910b [ 2336.368862] RDX: 0000000000000038 RSI: 00007fff45ac5060 RDI: 00007fff45ac4c60 [ 2336.375822] RBP: 00007fff45ac5060 R08: 00007fe02c4b1228 R09: 00007fe02c4b1260 [ 2336.383359] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2336.387093] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2336.416084] INFO: task writemany:36106 blocked for more than 120 seconds. [ 2336.425455] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2336.427897] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2336.431161] task:writemany state:D stack:0 pid:36106 ppid:36100 flags:0x80000004 [ 2336.434438] Call Trace: [ 2336.435373] __schedule+0x351/0xcb0 [ 2336.436942] schedule+0xc0/0x180 [ 2336.438141] schedule_preempt_disabled+0x21/0x40 [ 2336.439736] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2336.441517] ? __might_sleep+0x59/0xc0 [ 2336.443034] down_write+0x80/0xd0 [ 2336.444215] do_unlinkat+0x184/0x460 [ 2336.445459] __x64_sys_unlink+0x37/0x50 [ 2336.447039] do_syscall_64+0xc1/0x440 [ 2336.448033] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2336.449382] RIP: 0033:0x7fe02c11910b [ 2336.450744] Code: Unable to access opcode bytes at RIP 0x7fe02c1190e1. [ 2336.452643] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2336.455025] RAX: ffffffffffffffda RBX: 0000000000c97218 RCX: 00007fe02c11910b [ 2336.458694] RDX: 0000000000000038 RSI: 00007fff45ac5060 RDI: 00007fff45ac4c60 [ 2336.461316] RBP: 00007fff45ac5060 R08: 00007fe02c4b122c R09: 00007fe02c4b1260 [ 2336.463694] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2336.466355] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2340.057023] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2340.060247] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2340.060826] Modules linked in: lustre(O) [ 2340.075464] Modules linked in: lustre(O) [ 2340.077356] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2340.077430] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2340.077437] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2340.077440] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2340.077454] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2340.077458] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2340.077462] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2340.077465] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 2340.077467] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2340.077469] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2340.077471] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2340.077478] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2340.077482] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2340.077483] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2340.077491] Call Trace: [ 2340.077496] [ 2340.077502] ? show_regs.cold.9+0x22/0x2f [ 2340.077511] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2340.077517] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2340.077521] ? watchdog+0x40/0x40 [ 2340.077525] ? hrtimer_interrupt+0x128/0x2c0 [ 2340.077528] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2340.077535] ? apic_timer_interrupt+0xf/0x20 [ 2340.077539] [ 2340.077541] ? native_safe_halt+0x2e/0x50 [ 2340.077545] kvm_wait+0xcd/0xe0 [ 2340.077552] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2340.077558] do_raw_spin_lock+0x177/0x190 [ 2340.077562] _raw_spin_lock+0x21/0x30 [ 2340.077566] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2340.077867] ? do_raw_spin_unlock+0x75/0x190 [ 2340.077872] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2340.078070] ? do_wait_intr+0xf0/0xf0 [ 2340.080156] osp(O) [ 2340.098743] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2340.104138] ofd(O) [ 2340.108226] mdc_reint+0x2d/0x110 [mdc] [ 2340.108277] mdc_unlink+0x658/0x870 [mdc] [ 2340.108304] lmv_unlink+0x34d/0xbd0 [lmv] [ 2340.108334] ll_unlink+0x24d/0x7f0 [lustre] [ 2340.112648] lod(O) [ 2340.120905] vfs_unlink+0x202/0x330 [ 2340.126856] mdt(O) [ 2340.130495] do_unlinkat+0x2fa/0x460 [ 2340.135884] mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2340.135959] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2340.135967] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2340.135970] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2340.136042] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2340.136046] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2340.136051] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2340.136053] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2340.136054] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2340.136056] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2340.136059] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2340.136062] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2340.136064] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2340.136067] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2340.136076] Call Trace: [ 2340.136082] [ 2340.136087] ? show_regs.cold.9+0x22/0x2f [ 2340.136097] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2340.136103] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2340.136107] ? watchdog+0x40/0x40 [ 2340.136111] ? hrtimer_interrupt+0x128/0x2c0 [ 2340.136115] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2340.136121] ? apic_timer_interrupt+0xf/0x20 [ 2340.136125] [ 2340.136128] ? native_safe_halt+0x2e/0x50 [ 2340.136131] kvm_wait+0xcd/0xe0 [ 2340.136139] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2340.136145] do_raw_spin_lock+0x177/0x190 [ 2340.136149] _raw_spin_lock+0x21/0x30 [ 2340.136154] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2340.136446] obd_mod_free+0x46/0x140 [ptlrpc] [ 2340.136657] mdc_commit_open+0x91/0xb0 [mdc] [ 2340.136692] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2340.136886] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2340.141433] __x64_sys_unlink+0x37/0x50 [ 2340.141449] do_syscall_64+0xc1/0x440 [ 2340.141455] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2340.141464] RIP: 0033:0x7fe02c11910b [ 2340.141471] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2340.141475] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2340.141481] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2340.141483] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2340.146325] after_reply+0xe2c/0x1780 [ptlrpc] [ 2340.150474] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2340.150479] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2340.150482] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2340.377258] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2340.379463] ? __next_timer_interrupt+0x160/0x160 [ 2340.381441] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2340.383633] ? do_wait_intr+0xf0/0xf0 [ 2340.388837] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2340.391403] kthread+0x1d1/0x200 [ 2340.392958] ? set_kthread_struct+0x70/0x70 [ 2340.395321] ret_from_fork+0x1f/0x30 [ 2364.064010] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2364.073960] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2364.099057] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2364.116638] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2364.126065] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2364.132644] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2364.144494] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2364.163542] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2364.172144] RAX: 0000000000004b1a RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2364.183123] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2364.191758] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2364.198220] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2364.202420] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2364.210684] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2364.215051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2364.218046] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2364.221707] Call Trace: [ 2364.222724] [ 2364.226540] ? show_regs.cold.9+0x22/0x2f [ 2364.227983] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2364.231729] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2364.235288] ? watchdog+0x40/0x40 [ 2364.240939] ? hrtimer_interrupt+0x128/0x2c0 [ 2364.242249] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2364.246226] ? apic_timer_interrupt+0xf/0x20 [ 2364.248783] [ 2364.250361] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2364.252746] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2364.255758] do_raw_spin_lock+0x177/0x190 [ 2364.257943] _raw_spin_lock+0x21/0x30 [ 2364.259857] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2364.263242] process_one_work+0x2c8/0x700 [ 2364.265117] worker_thread+0x296/0x6e0 [ 2364.267113] ? rescuer_thread+0x570/0x570 [ 2364.269206] kthread+0x1d1/0x200 [ 2364.271111] ? set_kthread_struct+0x70/0x70 [ 2364.273229] ret_from_fork+0x1f/0x30 [ 2368.057026] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2368.060036] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2368.061583] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2368.084908] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2368.088335] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2368.088420] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2368.088429] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2368.088432] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2368.088449] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2368.088453] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2368.088458] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2368.088460] RDX: 0000000000001fa6 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2368.088462] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2368.088465] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2368.088467] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2368.088471] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2368.088474] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2368.088476] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2368.089608] Call Trace: [ 2368.089622] [ 2368.089630] ? show_regs.cold.9+0x22/0x2f [ 2368.089646] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2368.089654] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2368.089660] ? watchdog+0x40/0x40 [ 2368.089664] ? hrtimer_interrupt+0x128/0x2c0 [ 2368.089668] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2368.089679] ? apic_timer_interrupt+0xf/0x20 [ 2368.089683] [ 2368.089687] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2368.089693] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2368.089697] do_raw_spin_lock+0x177/0x190 [ 2368.089702] _raw_spin_lock+0x21/0x30 [ 2368.089710] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2368.090422] obd_mod_free+0x46/0x140 [ptlrpc] [ 2368.099263] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2368.099277] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2368.099296] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2368.099300] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2368.099307] RAX: 0000000000005670 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2368.099310] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2368.099312] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2368.099314] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2368.099317] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2368.099321] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2368.099324] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2368.099326] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2368.099334] Call Trace: [ 2368.099341] [ 2368.099349] ? show_regs.cold.9+0x22/0x2f [ 2368.099362] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2368.099370] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2368.099373] ? watchdog+0x40/0x40 [ 2368.099377] ? hrtimer_interrupt+0x128/0x2c0 [ 2368.099380] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2368.099390] ? apic_timer_interrupt+0xf/0x20 [ 2368.099394] [ 2368.099396] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2368.099401] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2368.099406] do_raw_spin_lock+0x177/0x190 [ 2368.099411] _raw_spin_lock+0x21/0x30 [ 2368.099418] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2368.099733] ? do_raw_spin_unlock+0x75/0x190 [ 2368.099739] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2368.100048] ? do_wait_intr+0xf0/0xf0 [ 2368.100057] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2368.100259] mdc_reint+0x2d/0x110 [mdc] [ 2368.100300] mdc_unlink+0x658/0x870 [mdc] [ 2368.100329] lmv_unlink+0x34d/0xbd0 [lmv] [ 2368.100367] ll_unlink+0x24d/0x7f0 [lustre] [ 2368.101474] vfs_unlink+0x202/0x330 [ 2368.101555] do_unlinkat+0x2fa/0x460 [ 2368.101562] __x64_sys_unlink+0x37/0x50 [ 2368.101565] do_syscall_64+0xc1/0x440 [ 2368.101573] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2368.101582] RIP: 0033:0x7fe02c11910b [ 2368.101590] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2368.101594] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2368.101600] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2368.101602] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2368.101604] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2368.101607] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2368.101609] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2368.552425] mdc_commit_open+0x91/0xb0 [mdc] [ 2368.556930] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2368.562444] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2368.569534] after_reply+0xe2c/0x1780 [ptlrpc] [ 2368.571701] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2368.573461] ? __next_timer_interrupt+0x160/0x160 [ 2368.577805] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2368.582686] ? do_wait_intr+0xf0/0xf0 [ 2368.584297] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2368.587382] kthread+0x1d1/0x200 [ 2368.594547] ? set_kthread_struct+0x70/0x70 [ 2368.597272] ret_from_fork+0x1f/0x30 [ 2382.100095] rcu: INFO: rcu_sched self-detected stall on CPU [ 2382.102926] rcu: 1-....: (145612 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=43610 [ 2382.107557] (t=240008 jiffies g=278549 q=1254) [ 2382.109996] Sending NMI from CPU 1 to CPUs 0: [ 2382.112429] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 [ 2382.116096] NMI backtrace for cpu 1 [ 2382.120644] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2382.128380] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2382.133168] Call Trace: [ 2382.134665] [ 2382.135840] ? dump_stack+0xbb/0x10e [ 2382.137678] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 2382.140293] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 2382.142791] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 2382.145341] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 2382.147925] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 2382.150757] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 2382.152821] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 2382.155371] ? update_process_times+0x6d/0xc0 [ 2382.157537] ? tick_sched_handle.isra.18+0x27/0x90 [ 2382.160133] ? tick_sched_timer+0x53/0xb0 [ 2382.161489] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2382.163755] ? tick_sched_handle.isra.18+0x90/0x90 [ 2382.165961] ? hrtimer_interrupt+0x128/0x2c0 [ 2382.168196] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2382.170741] ? apic_timer_interrupt+0xf/0x20 [ 2382.173168] [ 2382.174352] ? native_safe_halt+0x2e/0x50 [ 2382.176378] ? kvm_wait+0xcd/0xe0 [ 2382.177994] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2382.180521] ? do_raw_spin_lock+0x177/0x190 [ 2382.182625] ? _raw_spin_lock+0x21/0x30 [ 2382.184662] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2382.187694] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2382.190334] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2382.193433] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2382.196158] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2382.199727] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2382.202664] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2382.205563] ? __next_timer_interrupt+0x160/0x160 [ 2382.208127] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2382.210534] ? do_wait_intr+0xf0/0xf0 [ 2382.212509] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2382.215783] ? kthread+0x1d1/0x200 [ 2382.217736] ? set_kthread_struct+0x70/0x70 [ 2382.219846] ? ret_from_fork+0x1f/0x30 [ 2392.064030] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2392.068161] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2392.108493] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2392.113543] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2392.127521] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2392.134902] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2392.137310] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2392.158384] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2392.164318] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2392.167451] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2392.170185] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2392.178220] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2392.181748] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2392.189995] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2392.198332] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2392.201123] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2392.210404] Call Trace: [ 2392.211756] [ 2392.212898] ? show_regs.cold.9+0x22/0x2f [ 2392.215269] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2392.217506] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2392.220196] ? watchdog+0x40/0x40 [ 2392.222543] ? hrtimer_interrupt+0x128/0x2c0 [ 2392.228482] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2392.230523] ? apic_timer_interrupt+0xf/0x20 [ 2392.236602] [ 2392.242443] ? native_safe_halt+0x2e/0x50 [ 2392.245658] kvm_wait+0xcd/0xe0 [ 2392.258147] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2392.262894] do_raw_spin_lock+0x177/0x190 [ 2392.264230] _raw_spin_lock+0x21/0x30 [ 2392.265417] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2392.273534] process_one_work+0x2c8/0x700 [ 2392.280792] worker_thread+0x296/0x6e0 [ 2392.286848] ? rescuer_thread+0x570/0x570 [ 2392.290994] kthread+0x1d1/0x200 [ 2392.295816] ? set_kthread_struct+0x70/0x70 [ 2392.297143] ret_from_fork+0x1f/0x30 [ 2408.057027] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2408.060138] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2408.065108] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2408.065190] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2408.065198] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2408.065201] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2408.065215] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2408.065220] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 2408.065224] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2408.065226] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 2408.065229] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2408.065231] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2408.065233] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2408.065237] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2408.065239] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2408.065242] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2408.065250] Call Trace: [ 2408.065256] [ 2408.065361] ? show_regs.cold.9+0x22/0x2f [ 2408.065373] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2408.065380] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2408.065385] ? watchdog+0x40/0x40 [ 2408.065389] ? hrtimer_interrupt+0x128/0x2c0 [ 2408.065400] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2408.065416] ? apic_timer_interrupt+0xf/0x20 [ 2408.065421] [ 2408.065424] ? native_safe_halt+0x2e/0x50 [ 2408.065428] kvm_wait+0xcd/0xe0 [ 2408.065440] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2408.065448] do_raw_spin_lock+0x177/0x190 [ 2408.065456] _raw_spin_lock+0x21/0x30 [ 2408.065462] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2408.065735] ? do_raw_spin_unlock+0x75/0x190 [ 2408.065741] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2408.066109] ? do_wait_intr+0xf0/0xf0 [ 2408.066117] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2408.066322] mdc_reint+0x2d/0x110 [mdc] [ 2408.066359] mdc_unlink+0x658/0x870 [mdc] [ 2408.066387] lmv_unlink+0x34d/0xbd0 [lmv] [ 2408.066456] ll_unlink+0x24d/0x7f0 [lustre] [ 2408.066576] vfs_unlink+0x202/0x330 [ 2408.066583] do_unlinkat+0x2fa/0x460 [ 2408.066588] __x64_sys_unlink+0x37/0x50 [ 2408.066592] do_syscall_64+0xc1/0x440 [ 2408.066599] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2408.066605] RIP: 0033:0x7fe02c11910b [ 2408.066611] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2408.066614] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2408.066619] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2408.066622] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2408.066624] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2408.066626] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2408.066628] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2408.239518] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2408.262289] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2408.267576] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2408.271880] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2408.274166] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2408.282546] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2408.286636] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2408.290285] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2408.293381] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2408.296263] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2408.299668] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2408.302666] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2408.306379] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2408.308990] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2408.312494] Call Trace: [ 2408.313499] [ 2408.314371] ? show_regs.cold.9+0x22/0x2f [ 2408.316066] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2408.317906] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2408.319683] ? watchdog+0x40/0x40 [ 2408.321062] ? hrtimer_interrupt+0x128/0x2c0 [ 2408.322751] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2408.324599] ? apic_timer_interrupt+0xf/0x20 [ 2408.326849] [ 2408.327691] ? native_safe_halt+0x2e/0x50 [ 2408.329222] kvm_wait+0xcd/0xe0 [ 2408.330490] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2408.332698] do_raw_spin_lock+0x177/0x190 [ 2408.334356] _raw_spin_lock+0x21/0x30 [ 2408.335936] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2408.338281] obd_mod_free+0x46/0x140 [ptlrpc] [ 2408.340373] mdc_commit_open+0x91/0xb0 [mdc] [ 2408.341963] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2408.344098] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2408.346315] after_reply+0xe2c/0x1780 [ptlrpc] [ 2408.348809] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2408.350820] ? __next_timer_interrupt+0x160/0x160 [ 2408.352877] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2408.354696] ? do_wait_intr+0xf0/0xf0 [ 2408.356067] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2408.358449] kthread+0x1d1/0x200 [ 2408.359925] ? set_kthread_struct+0x70/0x70 [ 2408.361699] ret_from_fork+0x1f/0x30 [ 2420.067023] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2420.080370] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2420.141054] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2420.155127] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2420.179078] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2420.188596] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2420.204186] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2420.254271] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2420.262817] RAX: 00000000000010b2 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2420.268982] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2420.277879] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2420.288946] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2420.305071] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2420.316894] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2420.328908] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2420.333230] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2420.336421] Call Trace: [ 2420.339739] [ 2420.342046] ? show_regs.cold.9+0x22/0x2f [ 2420.345974] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2420.351947] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2420.361580] ? watchdog+0x40/0x40 [ 2420.364206] ? hrtimer_interrupt+0x128/0x2c0 [ 2420.367012] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2420.373463] ? apic_timer_interrupt+0xf/0x20 [ 2420.382402] [ 2420.385702] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2420.392642] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2420.401765] do_raw_spin_lock+0x177/0x190 [ 2420.407530] _raw_spin_lock+0x21/0x30 [ 2420.413739] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2420.421866] process_one_work+0x2c8/0x700 [ 2420.425032] worker_thread+0x296/0x6e0 [ 2420.432203] ? rescuer_thread+0x570/0x570 [ 2420.437169] kthread+0x1d1/0x200 [ 2420.443822] ? set_kthread_struct+0x70/0x70 [ 2420.449088] ret_from_fork+0x1f/0x30 [ 2436.058587] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2436.060022] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2436.067865] Modules linked in: [ 2436.071237] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2436.075100] lustre(O) [ 2436.119493] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2436.119506] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2436.119509] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2436.119527] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2436.119531] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2436.119537] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2436.119540] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2436.119543] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2436.119545] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2436.119548] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2436.119555] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2436.119559] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2436.119561] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2436.119569] Call Trace: [ 2436.119574] [ 2436.119580] ? show_regs.cold.9+0x22/0x2f [ 2436.119598] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2436.119604] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2436.119609] ? watchdog+0x40/0x40 [ 2436.119614] ? hrtimer_interrupt+0x128/0x2c0 [ 2436.119618] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2436.119630] ? apic_timer_interrupt+0xf/0x20 [ 2436.119635] [ 2436.119639] ? native_safe_halt+0x2e/0x50 [ 2436.119643] kvm_wait+0xcd/0xe0 [ 2436.119651] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2436.119662] do_raw_spin_lock+0x177/0x190 [ 2436.119668] _raw_spin_lock+0x21/0x30 [ 2436.119675] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2436.121471] obd_mod_free+0x46/0x140 [ptlrpc] [ 2436.122644] mdc_commit_open+0x91/0xb0 [mdc] [ 2436.122680] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2436.128236] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2436.128375] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2436.128381] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2436.128384] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2436.128397] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2436.128401] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2436.128405] RAX: 00000000000021b4 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2436.128407] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2436.128410] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2436.128412] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2436.128414] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2436.128417] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2436.128420] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2436.128422] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2436.128431] Call Trace: [ 2436.128436] [ 2436.128441] ? show_regs.cold.9+0x22/0x2f [ 2436.128451] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2436.128458] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2436.128463] ? watchdog+0x40/0x40 [ 2436.128466] ? hrtimer_interrupt+0x128/0x2c0 [ 2436.128470] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2436.128478] ? apic_timer_interrupt+0xf/0x20 [ 2436.128483] [ 2436.128486] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2436.128491] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2436.128495] do_raw_spin_lock+0x177/0x190 [ 2436.128499] _raw_spin_lock+0x21/0x30 [ 2436.128505] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2436.128961] ? do_raw_spin_unlock+0x75/0x190 [ 2436.128968] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2436.129188] ? do_wait_intr+0xf0/0xf0 [ 2436.129195] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2436.129908] mdc_reint+0x2d/0x110 [mdc] [ 2436.129954] mdc_unlink+0x658/0x870 [mdc] [ 2436.129982] lmv_unlink+0x34d/0xbd0 [lmv] [ 2436.130229] ll_unlink+0x24d/0x7f0 [lustre] [ 2436.130421] vfs_unlink+0x202/0x330 [ 2436.130430] do_unlinkat+0x2fa/0x460 [ 2436.130435] __x64_sys_unlink+0x37/0x50 [ 2436.130439] do_syscall_64+0xc1/0x440 [ 2436.130444] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2436.130451] RIP: 0033:0x7fe02c11910b [ 2436.130458] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2436.130462] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2436.130467] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2436.130470] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2436.130472] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2436.130474] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2436.130476] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2436.663660] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2436.671390] after_reply+0xe2c/0x1780 [ptlrpc] [ 2436.680570] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2436.685768] ? __next_timer_interrupt+0x160/0x160 [ 2436.689744] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2436.697720] ? do_wait_intr+0xf0/0xf0 [ 2436.701281] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2436.703381] kthread+0x1d1/0x200 [ 2436.705164] ? set_kthread_struct+0x70/0x70 [ 2436.707032] ret_from_fork+0x1f/0x30 [ 2448.064518] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2448.067856] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2448.102795] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2448.108549] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2448.113433] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2448.117963] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2448.119477] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2448.125625] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2448.128937] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2448.130484] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2448.132967] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2448.134549] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2448.137720] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2448.141984] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2448.146868] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2448.152622] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2448.156080] Call Trace: [ 2448.157046] [ 2448.158081] ? show_regs.cold.9+0x22/0x2f [ 2448.159692] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2448.164901] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2448.170743] ? watchdog+0x40/0x40 [ 2448.173200] ? hrtimer_interrupt+0x128/0x2c0 [ 2448.176768] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2448.182291] ? apic_timer_interrupt+0xf/0x20 [ 2448.183859] [ 2448.184646] ? native_safe_halt+0x2e/0x50 [ 2448.186905] kvm_wait+0xcd/0xe0 [ 2448.188324] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2448.191503] do_raw_spin_lock+0x177/0x190 [ 2448.193512] _raw_spin_lock+0x21/0x30 [ 2448.196258] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2448.200027] process_one_work+0x2c8/0x700 [ 2448.204119] worker_thread+0x296/0x6e0 [ 2448.206030] ? rescuer_thread+0x570/0x570 [ 2448.208069] kthread+0x1d1/0x200 [ 2448.209258] ? set_kthread_struct+0x70/0x70 [ 2448.211647] ret_from_fork+0x1f/0x30 [ 2458.593931] INFO: task khugepaged:48 blocked for more than 120 seconds. [ 2458.597953] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2458.606802] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2458.615791] task:khugepaged state:D stack:0 pid:48 ppid:2 flags:0x80004000 [ 2458.625955] Call Trace: [ 2458.630330] __schedule+0x351/0xcb0 [ 2458.633872] ? wait_for_completion+0xae/0x1e0 [ 2458.639788] schedule+0xc0/0x180 [ 2458.643619] schedule_timeout+0x126/0x190 [ 2458.647489] ? try_to_wake_up+0x80/0x710 [ 2458.651032] ? __prepare_to_swait+0x5b/0x90 [ 2458.655376] ? do_raw_spin_unlock+0x75/0x190 [ 2458.657564] wait_for_completion+0xf0/0x1e0 [ 2458.660095] flush_work+0x1f3/0x390 [ 2458.661429] ? flush_workqueue_prep_pwqs+0x200/0x200 [ 2458.663422] ? do_raw_spin_unlock+0x75/0x190 [ 2458.665156] ? __pagevec_release+0x50/0x50 [ 2458.670783] lru_add_drain_all+0x1d4/0x280 [ 2458.672757] khugepaged+0xa2/0x16f0 [ 2458.676500] ? woken_wake_function+0x30/0x30 [ 2458.683531] ? collapse_pte_mapped_thp+0x8a0/0x8a0 [ 2458.685556] kthread+0x1d1/0x200 [ 2458.686665] ? set_kthread_struct+0x70/0x70 [ 2458.694638] ret_from_fork+0x1f/0x30 [ 2458.696537] INFO: task writemany:36103 blocked for more than 120 seconds. [ 2458.703719] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2458.707730] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2458.713684] task:writemany state:D stack:0 pid:36103 ppid:36100 flags:0x80000004 [ 2458.730662] Call Trace: [ 2458.732185] __schedule+0x351/0xcb0 [ 2458.736159] schedule+0xc0/0x180 [ 2458.737278] schedule_preempt_disabled+0x21/0x40 [ 2458.743104] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2458.745251] ? lprocfs_counter_add+0x15b/0x210 [obdclass] [ 2458.747762] down_write+0x80/0xd0 [ 2458.752762] do_last+0x2eb/0xfc0 [ 2458.754599] ? nd_jump_root+0xe5/0x160 [ 2458.762137] ? path_init+0x437/0x520 [ 2458.769157] path_openat+0xf7/0x500 [ 2458.770794] ? do_raw_spin_unlock+0x75/0x190 [ 2458.774305] do_filp_open+0x99/0x140 [ 2458.776369] ? getname_flags+0x6e/0x330 [ 2458.779231] ? __check_object_size+0xff/0x256 [ 2458.783540] ? do_raw_spin_unlock+0x75/0x190 [ 2458.786302] ? _raw_spin_unlock+0x12/0x30 [ 2458.791189] do_sys_openat2+0x2b4/0x410 [ 2458.792869] do_sys_open+0x73/0xa0 [ 2458.794718] __x64_sys_openat+0x24/0x30 [ 2458.800660] do_syscall_64+0xc1/0x440 [ 2458.802714] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2458.806756] RIP: 0033:0x7fe02c20e332 [ 2458.810975] Code: Unable to access opcode bytes at RIP 0x7fe02c20e308. [ 2458.813097] RSP: 002b:00007fff45ac4b90 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 [ 2458.819490] RAX: ffffffffffffffda RBX: 00000000000191d8 RCX: 00007fe02c20e332 [ 2458.831150] RDX: 0000000000000042 RSI: 00007fff45ac4c60 RDI: 00000000ffffff9c [ 2458.840096] RBP: 00007fff45ac5060 R08: 0000000000000000 R09: 0000000000000000 [ 2458.844697] R10: 00000000000001b6 R11: 0000000000000246 R12: 00007fff45ac4c60 [ 2458.849368] R13: 00000000ffffffff R14: 0000000000000191 R15: 0000000000000000 [ 2458.854124] INFO: task writemany:36104 blocked for more than 120 seconds. [ 2458.860130] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2458.864675] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2458.867706] task:writemany state:D stack:0 pid:36104 ppid:36100 flags:0x80000004 [ 2458.875691] Call Trace: [ 2458.876837] __schedule+0x351/0xcb0 [ 2458.886822] schedule+0xc0/0x180 [ 2458.888793] schedule_preempt_disabled+0x21/0x40 [ 2458.894360] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2458.896718] ? lprocfs_counter_add+0x15b/0x210 [obdclass] [ 2458.899706] down_write+0x80/0xd0 [ 2458.901831] do_last+0x2eb/0xfc0 [ 2458.903251] ? nd_jump_root+0xe5/0x160 [ 2458.907536] ? path_init+0x437/0x520 [ 2458.909277] path_openat+0xf7/0x500 [ 2458.911236] ? do_raw_spin_unlock+0x75/0x190 [ 2458.918366] do_filp_open+0x99/0x140 [ 2458.919711] ? getname_flags+0x6e/0x330 [ 2458.920939] ? __check_object_size+0xff/0x256 [ 2458.926139] ? do_raw_spin_unlock+0x75/0x190 [ 2458.929789] ? _raw_spin_unlock+0x12/0x30 [ 2458.935302] do_sys_openat2+0x2b4/0x410 [ 2458.937252] do_sys_open+0x73/0xa0 [ 2458.943459] __x64_sys_openat+0x24/0x30 [ 2458.949315] do_syscall_64+0xc1/0x440 [ 2458.952785] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2458.962150] RIP: 0033:0x7fe02c20e332 [ 2458.964114] Code: Unable to access opcode bytes at RIP 0x7fe02c20e308. [ 2458.970138] RSP: 002b:00007fff45ac4b90 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 [ 2458.974689] RAX: ffffffffffffffda RBX: 00000000000f0290 RCX: 00007fe02c20e332 [ 2458.977188] RDX: 0000000000000042 RSI: 00007fff45ac4c60 RDI: 00000000ffffff9c [ 2458.981577] RBP: 00007fff45ac5060 R08: 0000000000000000 R09: 0000000000000000 [ 2458.989501] R10: 00000000000001b6 R11: 0000000000000246 R12: 00007fff45ac4c60 [ 2458.993702] R13: 00000000ffffffff R14: 000000000000015f R15: 0000000000000000 [ 2459.001046] INFO: task writemany:36105 blocked for more than 120 seconds. [ 2459.004238] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2459.009059] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2459.017108] task:writemany state:D stack:0 pid:36105 ppid:36100 flags:0x80000004 [ 2459.021854] Call Trace: [ 2459.029892] __schedule+0x351/0xcb0 [ 2459.039728] schedule+0xc0/0x180 [ 2459.045497] schedule_preempt_disabled+0x21/0x40 [ 2459.048666] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2459.051051] ? __might_sleep+0x59/0xc0 [ 2459.053274] down_write+0x80/0xd0 [ 2459.055284] do_unlinkat+0x184/0x460 [ 2459.060548] __x64_sys_unlink+0x37/0x50 [ 2459.062853] do_syscall_64+0xc1/0x440 [ 2459.063990] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2459.066502] RIP: 0033:0x7fe02c11910b [ 2459.068588] Code: Unable to access opcode bytes at RIP 0x7fe02c1190e1. [ 2459.070767] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2459.074326] RAX: ffffffffffffffda RBX: 000000000058d508 RCX: 00007fe02c11910b [ 2459.083039] RDX: 0000000000000038 RSI: 00007fff45ac5060 RDI: 00007fff45ac4c60 [ 2459.091294] RBP: 00007fff45ac5060 R08: 00007fe02c4b1228 R09: 00007fe02c4b1260 [ 2459.093691] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2459.098017] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2459.113415] INFO: task writemany:36106 blocked for more than 120 seconds. [ 2459.118484] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2459.121218] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2459.128860] task:writemany state:D stack:0 pid:36106 ppid:36100 flags:0x80000004 [ 2459.134752] Call Trace: [ 2459.136175] __schedule+0x351/0xcb0 [ 2459.137703] schedule+0xc0/0x180 [ 2459.142046] schedule_preempt_disabled+0x21/0x40 [ 2459.147066] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2459.148059] ? __might_sleep+0x59/0xc0 [ 2459.150127] down_write+0x80/0xd0 [ 2459.152053] do_unlinkat+0x184/0x460 [ 2459.153592] __x64_sys_unlink+0x37/0x50 [ 2459.155203] do_syscall_64+0xc1/0x440 [ 2459.161201] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2459.166333] RIP: 0033:0x7fe02c11910b [ 2459.173469] Code: Unable to access opcode bytes at RIP 0x7fe02c1190e1. [ 2459.182340] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2459.185260] RAX: ffffffffffffffda RBX: 0000000000c97218 RCX: 00007fe02c11910b [ 2459.192820] RDX: 0000000000000038 RSI: 00007fff45ac5060 RDI: 00007fff45ac4c60 [ 2459.196948] RBP: 00007fff45ac5060 R08: 00007fe02c4b122c R09: 00007fe02c4b1260 [ 2459.199135] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2459.214454] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2464.057160] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2464.061644] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2464.064383] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2464.092021] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2464.099755] Modules linked in: [ 2464.105678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2464.105688] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2464.105703] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2464.105707] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 2464.105713] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2464.105715] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 2464.105717] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2464.105720] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2464.105722] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2464.105725] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2464.105729] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2464.105731] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2464.105740] Call Trace: [ 2464.105745] [ 2464.105750] ? show_regs.cold.9+0x22/0x2f [ 2464.105759] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2464.105767] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2464.105772] ? watchdog+0x40/0x40 [ 2464.105776] ? hrtimer_interrupt+0x128/0x2c0 [ 2464.105780] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2464.105787] ? apic_timer_interrupt+0xf/0x20 [ 2464.105792] [ 2464.105795] ? native_safe_halt+0x2e/0x50 [ 2464.105799] kvm_wait+0xcd/0xe0 [ 2464.105807] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2464.105814] do_raw_spin_lock+0x177/0x190 [ 2464.105817] _raw_spin_lock+0x21/0x30 [ 2464.105823] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2464.111919] lustre(O) [ 2464.116016] ? do_raw_spin_unlock+0x75/0x190 [ 2464.118041] osp(O) [ 2464.128512] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2464.140400] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2464.140483] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2464.140491] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2464.140493] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2464.145051] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2464.145296] ? do_wait_intr+0xf0/0xf0 [ 2464.148765] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 [ 2464.153242] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2464.153541] mdc_reint+0x2d/0x110 [mdc] [ 2464.153583] mdc_unlink+0x658/0x870 [mdc] [ 2464.153611] lmv_unlink+0x34d/0xbd0 [lmv] [ 2464.153641] ll_unlink+0x24d/0x7f0 [lustre] [ 2464.153856] vfs_unlink+0x202/0x330 [ 2464.153866] do_unlinkat+0x2fa/0x460 [ 2464.153872] __x64_sys_unlink+0x37/0x50 [ 2464.153875] do_syscall_64+0xc1/0x440 [ 2464.153882] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2464.153890] RIP: 0033:0x7fe02c11910b [ 2464.153898] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2464.153902] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2464.153907] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2464.153910] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2464.153912] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2464.153914] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2464.153916] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2464.401331] ORIG_RAX: ffffffffffffff13 [ 2464.403384] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2464.410865] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2464.414161] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2464.419100] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2464.422751] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2464.426949] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2464.434480] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2464.438946] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2464.445712] Call Trace: [ 2464.450270] [ 2464.451406] ? show_regs.cold.9+0x22/0x2f [ 2464.453797] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2464.463449] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2464.469009] ? watchdog+0x40/0x40 [ 2464.471190] ? hrtimer_interrupt+0x128/0x2c0 [ 2464.473210] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2464.476252] ? apic_timer_interrupt+0xf/0x20 [ 2464.478316] [ 2464.482113] ? native_safe_halt+0x2e/0x50 [ 2464.485384] kvm_wait+0xcd/0xe0 [ 2464.487089] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2464.489592] do_raw_spin_lock+0x177/0x190 [ 2464.491785] _raw_spin_lock+0x21/0x30 [ 2464.493058] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2464.496325] obd_mod_free+0x46/0x140 [ptlrpc] [ 2464.501435] mdc_commit_open+0x91/0xb0 [mdc] [ 2464.503186] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2464.507635] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2464.513177] after_reply+0xe2c/0x1780 [ptlrpc] [ 2464.516625] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2464.523252] ? __next_timer_interrupt+0x160/0x160 [ 2464.526574] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2464.532669] ? do_wait_intr+0xf0/0xf0 [ 2464.534361] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2464.539678] kthread+0x1d1/0x200 [ 2464.542039] ? set_kthread_struct+0x70/0x70 [ 2464.544127] ret_from_fork+0x1f/0x30 [ 2476.067088] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2476.074407] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2476.116083] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2476.128089] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2476.142046] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2476.154023] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2476.156095] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2476.180168] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2476.185746] RAX: 0000000000007a16 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2476.195079] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2476.199166] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2476.202680] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2476.207687] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2476.213945] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2476.224474] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2476.230054] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2476.233483] Call Trace: [ 2476.238951] [ 2476.241121] ? show_regs.cold.9+0x22/0x2f [ 2476.243150] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2476.251103] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2476.260224] ? watchdog+0x40/0x40 [ 2476.263787] ? hrtimer_interrupt+0x128/0x2c0 [ 2476.265321] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2476.273027] ? apic_timer_interrupt+0xf/0x20 [ 2476.278151] [ 2476.278968] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2476.284536] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2476.289642] do_raw_spin_lock+0x177/0x190 [ 2476.295512] _raw_spin_lock+0x21/0x30 [ 2476.296795] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2476.304408] process_one_work+0x2c8/0x700 [ 2476.306732] worker_thread+0x296/0x6e0 [ 2476.309500] ? rescuer_thread+0x570/0x570 [ 2476.313082] kthread+0x1d1/0x200 [ 2476.314522] ? set_kthread_struct+0x70/0x70 [ 2476.317411] ret_from_fork+0x1f/0x30 [ 2492.057025] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2492.060307] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2492.065860] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr [ 2492.072520] Modules linked in: [ 2492.072640] i2c_piix4 [ 2492.088781] lustre(O) [ 2492.091783] squashfs [ 2492.092872] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2492.092951] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2492.092958] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2492.093958] ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2492.093982] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2492.093988] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2492.093990] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2492.094842] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2492.094847] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2492.094852] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2492.094854] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 2492.094856] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2492.094858] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2492.094860] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2492.094864] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2492.094867] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2492.094869] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2492.094877] Call Trace: [ 2492.094921] [ 2492.094927] ? show_regs.cold.9+0x22/0x2f [ 2492.094944] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2492.094950] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2492.094954] ? watchdog+0x40/0x40 [ 2492.094958] ? hrtimer_interrupt+0x128/0x2c0 [ 2492.094962] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2492.094968] ? apic_timer_interrupt+0xf/0x20 [ 2492.094972] [ 2492.094975] ? native_safe_halt+0x2e/0x50 [ 2492.094979] kvm_wait+0xcd/0xe0 [ 2492.094985] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2492.094992] do_raw_spin_lock+0x177/0x190 [ 2492.094996] _raw_spin_lock+0x21/0x30 [ 2492.095010] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2492.095589] ? do_raw_spin_unlock+0x75/0x190 [ 2492.095595] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2492.095878] ? do_wait_intr+0xf0/0xf0 [ 2492.095885] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2492.096310] mdc_reint+0x2d/0x110 [mdc] [ 2492.096410] mdc_unlink+0x658/0x870 [mdc] [ 2492.096443] lmv_unlink+0x34d/0xbd0 [lmv] [ 2492.096472] ll_unlink+0x24d/0x7f0 [lustre] [ 2492.096656] vfs_unlink+0x202/0x330 [ 2492.096664] do_unlinkat+0x2fa/0x460 [ 2492.096670] __x64_sys_unlink+0x37/0x50 [ 2492.096673] do_syscall_64+0xc1/0x440 [ 2492.096680] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2492.096687] RIP: 0033:0x7fe02c11910b [ 2492.096694] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2492.096697] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2492.096702] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2492.096704] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2492.096706] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2492.096708] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2492.096709] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2492.329989] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2492.332597] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2492.340449] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2492.343466] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2492.346763] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2492.350604] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2492.354967] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2492.358982] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2492.362422] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2492.365732] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2492.368160] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2492.372307] Call Trace: [ 2492.373593] [ 2492.374577] ? show_regs.cold.9+0x22/0x2f [ 2492.376192] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2492.378368] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2492.379901] ? watchdog+0x40/0x40 [ 2492.380802] ? hrtimer_interrupt+0x128/0x2c0 [ 2492.381903] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2492.383944] ? apic_timer_interrupt+0xf/0x20 [ 2492.385684] [ 2492.386600] ? native_safe_halt+0x2e/0x50 [ 2492.387986] kvm_wait+0xcd/0xe0 [ 2492.389507] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2492.391624] do_raw_spin_lock+0x177/0x190 [ 2492.393549] _raw_spin_lock+0x21/0x30 [ 2492.395288] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2492.398651] obd_mod_free+0x46/0x140 [ptlrpc] [ 2492.400987] mdc_commit_open+0x91/0xb0 [mdc] [ 2492.403502] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2492.406162] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2492.411444] after_reply+0xe2c/0x1780 [ptlrpc] [ 2492.416993] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2492.424753] ? __next_timer_interrupt+0x160/0x160 [ 2492.429484] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2492.435417] ? do_wait_intr+0xf0/0xf0 [ 2492.440100] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2492.448738] kthread+0x1d1/0x200 [ 2492.452676] ? set_kthread_struct+0x70/0x70 [ 2492.455015] ret_from_fork+0x1f/0x30 [ 2504.065471] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2504.070543] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2504.106586] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2504.120004] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2504.129813] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2504.134538] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2504.137619] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2504.149141] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 2504.153553] RAX: 0000000000007a68 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2504.161807] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2504.170261] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2504.179537] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2504.184848] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2504.189323] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2504.194614] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2504.197817] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2504.201706] Call Trace: [ 2504.203572] [ 2504.205217] ? show_regs.cold.9+0x22/0x2f [ 2504.208093] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2504.211822] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2504.214352] ? watchdog+0x40/0x40 [ 2504.216168] ? hrtimer_interrupt+0x128/0x2c0 [ 2504.219193] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2504.223491] ? apic_timer_interrupt+0xf/0x20 [ 2504.226684] [ 2504.227898] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2504.231873] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2504.235273] do_raw_spin_lock+0x177/0x190 [ 2504.237310] _raw_spin_lock+0x21/0x30 [ 2504.239307] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2504.242703] process_one_work+0x2c8/0x700 [ 2504.244795] worker_thread+0x296/0x6e0 [ 2504.247301] ? rescuer_thread+0x570/0x570 [ 2504.249321] kthread+0x1d1/0x200 [ 2504.251467] ? set_kthread_struct+0x70/0x70 [ 2504.253457] ret_from_fork+0x1f/0x30 [ 2520.057009] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2520.062536] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2520.066287] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2520.092838] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2520.092854] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2520.092856] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2520.092872] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2520.092876] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2520.092881] RAX: 0000000000005d08 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2520.092884] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2520.092886] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2520.092888] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2520.092890] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2520.092894] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2520.092897] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2520.092900] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2520.092908] Call Trace: [ 2520.092914] [ 2520.092920] ? show_regs.cold.9+0x22/0x2f [ 2520.092929] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2520.092936] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2520.092940] ? watchdog+0x40/0x40 [ 2520.092944] ? hrtimer_interrupt+0x128/0x2c0 [ 2520.092948] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2520.092954] ? apic_timer_interrupt+0xf/0x20 [ 2520.092959] [ 2520.092962] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2520.092967] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2520.092973] do_raw_spin_lock+0x177/0x190 [ 2520.092978] _raw_spin_lock+0x21/0x30 [ 2520.092985] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2520.093480] ? do_raw_spin_unlock+0x75/0x190 [ 2520.093486] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2520.093895] ? do_wait_intr+0xf0/0xf0 [ 2520.093904] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2520.094770] mdc_reint+0x2d/0x110 [mdc] [ 2520.094808] mdc_unlink+0x658/0x870 [mdc] [ 2520.094836] lmv_unlink+0x34d/0xbd0 [lmv] [ 2520.094865] ll_unlink+0x24d/0x7f0 [lustre] [ 2520.096216] vfs_unlink+0x202/0x330 [ 2520.096227] do_unlinkat+0x2fa/0x460 [ 2520.096232] __x64_sys_unlink+0x37/0x50 [ 2520.096235] do_syscall_64+0xc1/0x440 [ 2520.096241] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2520.096256] RIP: 0033:0x7fe02c11910b [ 2520.096264] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2520.096267] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2520.096273] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2520.096275] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2520.096277] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2520.096279] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2520.096280] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2520.490289] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2520.539270] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2520.547535] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2520.561787] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2520.566235] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2520.581396] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2520.588798] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2520.592404] RDX: 000000000000271e RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2520.599426] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2520.606241] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2520.611148] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2520.620368] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2520.627135] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2520.636634] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2520.644023] Call Trace: [ 2520.645601] [ 2520.648191] ? show_regs.cold.9+0x22/0x2f [ 2520.651018] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2520.656070] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2520.657340] ? watchdog+0x40/0x40 [ 2520.664442] ? hrtimer_interrupt+0x128/0x2c0 [ 2520.674392] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2520.684938] ? apic_timer_interrupt+0xf/0x20 [ 2520.690238] [ 2520.694063] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2520.699112] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2520.706706] do_raw_spin_lock+0x177/0x190 [ 2520.713804] _raw_spin_lock+0x21/0x30 [ 2520.717181] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2520.723794] obd_mod_free+0x46/0x140 [ptlrpc] [ 2520.731193] mdc_commit_open+0x91/0xb0 [mdc] [ 2520.733678] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2520.737214] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2520.741308] after_reply+0xe2c/0x1780 [ptlrpc] [ 2520.745011] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2520.748206] ? __next_timer_interrupt+0x160/0x160 [ 2520.752026] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2520.758485] ? do_wait_intr+0xf0/0xf0 [ 2520.764136] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2520.775302] kthread+0x1d1/0x200 [ 2520.781839] ? set_kthread_struct+0x70/0x70 [ 2520.783389] ret_from_fork+0x1f/0x30 [ 2532.066266] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2532.078148] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2532.134097] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2532.154058] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2532.181806] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2532.184764] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2532.201429] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2532.220156] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2532.231719] RAX: 0000000000002a19 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2532.237888] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2532.242430] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2532.249942] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2532.258500] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2532.267446] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2532.281884] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2532.292175] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2532.301177] Call Trace: [ 2532.304937] [ 2532.305969] ? show_regs.cold.9+0x22/0x2f [ 2532.307716] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2532.313745] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2532.316331] ? watchdog+0x40/0x40 [ 2532.318652] ? hrtimer_interrupt+0x128/0x2c0 [ 2532.327311] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2532.329362] ? apic_timer_interrupt+0xf/0x20 [ 2532.331799] [ 2532.332862] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2532.335823] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2532.340311] do_raw_spin_lock+0x177/0x190 [ 2532.344102] _raw_spin_lock+0x21/0x30 [ 2532.346256] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2532.348375] process_one_work+0x2c8/0x700 [ 2532.362458] worker_thread+0x296/0x6e0 [ 2532.368554] ? rescuer_thread+0x570/0x570 [ 2532.372512] kthread+0x1d1/0x200 [ 2532.373620] ? set_kthread_struct+0x70/0x70 [ 2532.379037] ret_from_fork+0x1f/0x30 [ 2548.057286] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2548.066251] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2548.069803] Modules linked in: lustre(O) [ 2548.077203] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2548.077261] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2548.077266] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2548.077269] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2548.077280] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2548.077282] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2548.077286] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2548.077287] RDX: 000000000000793a RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2548.077289] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2548.077290] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2548.077292] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2548.077294] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2548.077296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2548.077298] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2548.077304] Call Trace: [ 2548.077309] [ 2548.077313] ? show_regs.cold.9+0x22/0x2f [ 2548.077320] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2548.077324] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2548.077327] ? watchdog+0x40/0x40 [ 2548.077329] ? hrtimer_interrupt+0x128/0x2c0 [ 2548.077332] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2548.077338] ? apic_timer_interrupt+0xf/0x20 [ 2548.077342] [ 2548.077343] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2548.077347] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2548.077350] do_raw_spin_lock+0x177/0x190 [ 2548.077353] _raw_spin_lock+0x21/0x30 [ 2548.077356] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2548.077624] obd_mod_free+0x46/0x140 [ptlrpc] [ 2548.077761] mdc_commit_open+0x91/0xb0 [mdc] [ 2548.077794] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2548.077925] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2548.078516] after_reply+0xe2c/0x1780 [ptlrpc] [ 2548.078692] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2548.084994] osp(O) [ 2548.125760] ? __next_timer_interrupt+0x160/0x160 [ 2548.126263] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2548.126605] ? do_wait_intr+0xf0/0xf0 [ 2548.126615] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2548.128118] kthread+0x1d1/0x200 [ 2548.128127] ? set_kthread_struct+0x70/0x70 [ 2548.128131] ret_from_fork+0x1f/0x30 [ 2548.306161] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2548.337414] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2548.346590] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2548.356180] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2548.364535] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2548.391682] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2548.398127] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2548.405694] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2548.408121] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2548.414730] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2548.426912] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2548.432604] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2548.441192] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2548.447502] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2548.461490] Call Trace: [ 2548.464442] [ 2548.465107] ? show_regs.cold.9+0x22/0x2f [ 2548.469189] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2548.471245] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2548.472874] ? watchdog+0x40/0x40 [ 2548.474196] ? hrtimer_interrupt+0x128/0x2c0 [ 2548.475889] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2548.477892] ? apic_timer_interrupt+0xf/0x20 [ 2548.479581] [ 2548.480419] ? native_safe_halt+0x2e/0x50 [ 2548.481911] kvm_wait+0xcd/0xe0 [ 2548.483084] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2548.485326] do_raw_spin_lock+0x177/0x190 [ 2548.493834] _raw_spin_lock+0x21/0x30 [ 2548.495271] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2548.497598] ? do_raw_spin_unlock+0x75/0x190 [ 2548.499005] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2548.501508] ? do_wait_intr+0xf0/0xf0 [ 2548.502891] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2548.505421] mdc_reint+0x2d/0x110 [mdc] [ 2548.506940] mdc_unlink+0x658/0x870 [mdc] [ 2548.508808] lmv_unlink+0x34d/0xbd0 [lmv] [ 2548.510754] ll_unlink+0x24d/0x7f0 [lustre] [ 2548.512797] vfs_unlink+0x202/0x330 [ 2548.514440] do_unlinkat+0x2fa/0x460 [ 2548.515902] __x64_sys_unlink+0x37/0x50 [ 2548.518567] do_syscall_64+0xc1/0x440 [ 2548.520133] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2548.522757] RIP: 0033:0x7fe02c11910b [ 2548.523632] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2548.535176] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2548.538268] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2548.541145] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2548.550682] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2548.559062] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2548.566107] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2560.064024] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2560.067146] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2560.083029] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2560.090283] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2560.094155] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2560.097266] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2560.099569] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2560.119925] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 2560.130728] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2560.139283] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2560.148923] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2560.156286] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2560.162548] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2560.166330] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2560.173914] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2560.182999] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2560.191102] Call Trace: [ 2560.195538] [ 2560.197657] ? show_regs.cold.9+0x22/0x2f [ 2560.201710] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2560.210290] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2560.217954] ? watchdog+0x40/0x40 [ 2560.219410] ? hrtimer_interrupt+0x128/0x2c0 [ 2560.220790] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2560.228758] ? apic_timer_interrupt+0xf/0x20 [ 2560.230279] [ 2560.233887] ? native_safe_halt+0x2e/0x50 [ 2560.236922] kvm_wait+0xcd/0xe0 [ 2560.239166] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2560.242483] do_raw_spin_lock+0x177/0x190 [ 2560.246318] _raw_spin_lock+0x21/0x30 [ 2560.251323] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2560.253587] process_one_work+0x2c8/0x700 [ 2560.260573] worker_thread+0x296/0x6e0 [ 2560.262067] ? rescuer_thread+0x570/0x570 [ 2560.271971] kthread+0x1d1/0x200 [ 2560.273149] ? set_kthread_struct+0x70/0x70 [ 2560.279276] ret_from_fork+0x1f/0x30 [ 2562.109568] rcu: INFO: rcu_sched self-detected stall on CPU [ 2562.112694] rcu: 1-....: (257989 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=76875 [ 2562.122616] (t=420017 jiffies g=278549 q=3951) [ 2562.128144] Sending NMI from CPU 1 to CPUs 0: [ 2562.132009] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 [ 2562.132763] NMI backtrace for cpu 1 [ 2562.142175] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2562.149076] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2562.153779] Call Trace: [ 2562.156557] [ 2562.157595] ? dump_stack+0xbb/0x10e [ 2562.160150] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 2562.162041] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 2562.164267] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 2562.166540] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 2562.170460] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 2562.172956] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 2562.174683] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 2562.178006] ? update_process_times+0x6d/0xc0 [ 2562.180456] ? tick_sched_handle.isra.18+0x27/0x90 [ 2562.182656] ? tick_sched_timer+0x53/0xb0 [ 2562.184469] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2562.186507] ? tick_sched_handle.isra.18+0x90/0x90 [ 2562.188554] ? hrtimer_interrupt+0x128/0x2c0 [ 2562.190973] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2562.193214] ? apic_timer_interrupt+0xf/0x20 [ 2562.195190] [ 2562.196324] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2562.198967] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2562.201285] ? do_raw_spin_lock+0x177/0x190 [ 2562.203286] ? _raw_spin_lock+0x21/0x30 [ 2562.205453] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2562.209030] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2562.211864] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2562.214138] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2562.217934] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2562.221784] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2562.224296] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2562.228508] ? __next_timer_interrupt+0x160/0x160 [ 2562.232636] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2562.235023] ? do_wait_intr+0xf0/0xf0 [ 2562.236477] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2562.239240] ? kthread+0x1d1/0x200 [ 2562.241276] ? set_kthread_struct+0x70/0x70 [ 2562.249265] ? ret_from_fork+0x1f/0x30 [ 2588.057072] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2588.060220] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2588.060982] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2588.061064] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2588.061070] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2588.061073] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2588.061086] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2588.061090] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2588.061094] RAX: 0000000000000d26 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2588.061097] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2588.061099] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2588.061101] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2588.061103] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2588.061107] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2588.061110] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2588.061112] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2588.061120] Call Trace: [ 2588.061125] [ 2588.061130] ? show_regs.cold.9+0x22/0x2f [ 2588.061139] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2588.061145] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2588.061149] ? watchdog+0x40/0x40 [ 2588.061153] ? hrtimer_interrupt+0x128/0x2c0 [ 2588.061157] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2588.061164] ? apic_timer_interrupt+0xf/0x20 [ 2588.061169] [ 2588.061173] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2588.061177] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2588.061182] do_raw_spin_lock+0x177/0x190 [ 2588.061187] _raw_spin_lock+0x21/0x30 [ 2588.061192] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2588.061494] ? do_raw_spin_unlock+0x75/0x190 [ 2588.061500] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2588.061995] ? do_wait_intr+0xf0/0xf0 [ 2588.062108] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2588.062329] mdc_reint+0x2d/0x110 [mdc] [ 2588.062372] mdc_unlink+0x658/0x870 [mdc] [ 2588.062398] lmv_unlink+0x34d/0xbd0 [lmv] [ 2588.062429] ll_unlink+0x24d/0x7f0 [lustre] [ 2588.062661] vfs_unlink+0x202/0x330 [ 2588.062670] do_unlinkat+0x2fa/0x460 [ 2588.062675] __x64_sys_unlink+0x37/0x50 [ 2588.062678] do_syscall_64+0xc1/0x440 [ 2588.062684] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2588.062691] RIP: 0033:0x7fe02c11910b [ 2588.062697] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2588.062701] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2588.062705] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2588.062707] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2588.062709] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2588.062711] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2588.062713] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2588.064062] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2588.074874] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2588.074977] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2588.074984] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2588.074986] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2588.074999] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2588.075008] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2588.075012] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2588.075015] RDX: 00000000000005e4 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2588.075018] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2588.075020] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2588.075023] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2588.075025] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2588.075028] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2588.075030] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2588.075039] Call Trace: [ 2588.075045] [ 2588.075050] ? show_regs.cold.9+0x22/0x2f [ 2588.075059] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2588.075065] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2588.075073] ? watchdog+0x40/0x40 [ 2588.075078] ? hrtimer_interrupt+0x128/0x2c0 [ 2588.075082] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2588.075090] ? apic_timer_interrupt+0xf/0x20 [ 2588.075095] [ 2588.075098] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2588.075105] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2588.075110] do_raw_spin_lock+0x177/0x190 [ 2588.075114] _raw_spin_lock+0x21/0x30 [ 2588.075119] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2588.075404] obd_mod_free+0x46/0x140 [ptlrpc] [ 2588.075594] mdc_commit_open+0x91/0xb0 [mdc] [ 2588.075626] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2588.076162] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2588.076378] after_reply+0xe2c/0x1780 [ptlrpc] [ 2588.076576] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2588.077120] ? __next_timer_interrupt+0x160/0x160 [ 2588.077129] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2588.077375] ? do_wait_intr+0xf0/0xf0 [ 2588.077383] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2588.077836] kthread+0x1d1/0x200 [ 2588.077848] ? set_kthread_struct+0x70/0x70 [ 2588.077852] ret_from_fork+0x1f/0x30 [ 2588.653863] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2588.699029] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2588.704970] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2588.710894] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2588.714123] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2588.716901] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2588.727248] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2588.731081] RAX: 0000000000001286 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2588.736549] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2588.741188] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2588.747500] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2588.751938] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2588.757594] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2588.763358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2588.766102] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2588.776254] Call Trace: [ 2588.777972] [ 2588.778951] ? show_regs.cold.9+0x22/0x2f [ 2588.785119] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2588.790154] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2588.791443] ? watchdog+0x40/0x40 [ 2588.796911] ? hrtimer_interrupt+0x128/0x2c0 [ 2588.799310] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2588.812060] ? apic_timer_interrupt+0xf/0x20 [ 2588.815129] [ 2588.816456] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2588.825042] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2588.829366] do_raw_spin_lock+0x177/0x190 [ 2588.832449] _raw_spin_lock+0x21/0x30 [ 2588.834188] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2588.837340] process_one_work+0x2c8/0x700 [ 2588.838967] worker_thread+0x296/0x6e0 [ 2588.841009] ? rescuer_thread+0x570/0x570 [ 2588.844495] kthread+0x1d1/0x200 [ 2588.845680] ? set_kthread_struct+0x70/0x70 [ 2588.855556] ret_from_fork+0x1f/0x30 [ 2616.057356] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2616.060670] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2616.065408] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2616.065479] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2616.065486] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2616.065489] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2616.065503] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2616.065507] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2616.065510] RAX: 0000000000007e26 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2616.065513] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2616.065514] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2616.065516] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2616.065518] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2616.065521] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2616.065524] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2616.065526] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2616.065534] Call Trace: [ 2616.065539] [ 2616.065544] ? show_regs.cold.9+0x22/0x2f [ 2616.065553] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2616.065559] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2616.065564] ? watchdog+0x40/0x40 [ 2616.065567] ? hrtimer_interrupt+0x128/0x2c0 [ 2616.065572] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2616.065580] ? apic_timer_interrupt+0xf/0x20 [ 2616.065584] [ 2616.065587] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2616.065592] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2616.065597] do_raw_spin_lock+0x177/0x190 [ 2616.065602] _raw_spin_lock+0x21/0x30 [ 2616.065607] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2616.068770] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2616.068856] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2616.068865] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2616.068868] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2616.068883] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2616.068888] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2616.068892] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2616.068895] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2616.068897] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2616.068899] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2616.068901] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2616.068904] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2616.068907] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2616.068910] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2616.068918] Call Trace: [ 2616.068924] [ 2616.068932] ? show_regs.cold.9+0x22/0x2f [ 2616.068945] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2616.068952] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2616.068957] ? watchdog+0x40/0x40 [ 2616.068962] ? hrtimer_interrupt+0x128/0x2c0 [ 2616.068966] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2616.068975] ? apic_timer_interrupt+0xf/0x20 [ 2616.068979] [ 2616.068982] ? native_safe_halt+0x2e/0x50 [ 2616.068985] kvm_wait+0xcd/0xe0 [ 2616.068996] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2616.069005] do_raw_spin_lock+0x177/0x190 [ 2616.069010] _raw_spin_lock+0x21/0x30 [ 2616.069015] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2616.069711] obd_mod_free+0x46/0x140 [ptlrpc] [ 2616.069919] mdc_commit_open+0x91/0xb0 [mdc] [ 2616.069969] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2616.070098] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2616.070114] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2616.070202] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2616.070169] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2616.070368] after_reply+0xe2c/0x1780 [ptlrpc] [ 2616.070563] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2616.070756] ? __next_timer_interrupt+0x160/0x160 [ 2616.070783] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2616.070983] ? do_wait_intr+0xf0/0xf0 [ 2616.070995] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2616.071194] kthread+0x1d1/0x200 [ 2616.071201] ? set_kthread_struct+0x70/0x70 [ 2616.071204] ret_from_fork+0x1f/0x30 [ 2616.132722] ? do_raw_spin_unlock+0x75/0x190 [ 2616.132743] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2616.140667] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2616.140682] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2616.141014] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2616.141028] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2616.141033] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2616.141038] RAX: 0000000000003d8e RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2616.141040] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2616.141042] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2616.141044] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2616.141047] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2616.141050] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2616.141053] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2616.141056] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2616.141065] Call Trace: [ 2616.141071] [ 2616.141075] ? show_regs.cold.9+0x22/0x2f [ 2616.141087] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2616.141094] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2616.141099] ? watchdog+0x40/0x40 [ 2616.141104] ? hrtimer_interrupt+0x128/0x2c0 [ 2616.141108] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2616.141118] ? apic_timer_interrupt+0xf/0x20 [ 2616.141127] [ 2616.141131] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2616.141136] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2616.141141] do_raw_spin_lock+0x177/0x190 [ 2616.141215] _raw_spin_lock+0x21/0x30 [ 2616.141227] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2616.141521] process_one_work+0x2c8/0x700 [ 2616.141530] worker_thread+0x296/0x6e0 [ 2616.141535] ? rescuer_thread+0x570/0x570 [ 2616.141539] kthread+0x1d1/0x200 [ 2616.141545] ? set_kthread_struct+0x70/0x70 [ 2616.141548] ret_from_fork+0x1f/0x30 [ 2616.868513] ? do_wait_intr+0xf0/0xf0 [ 2616.870336] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2616.872667] mdc_reint+0x2d/0x110 [mdc] [ 2616.874283] mdc_unlink+0x658/0x870 [mdc] [ 2616.876040] lmv_unlink+0x34d/0xbd0 [lmv] [ 2616.877810] ll_unlink+0x24d/0x7f0 [lustre] [ 2616.880298] vfs_unlink+0x202/0x330 [ 2616.881892] do_unlinkat+0x2fa/0x460 [ 2616.884216] __x64_sys_unlink+0x37/0x50 [ 2616.892981] do_syscall_64+0xc1/0x440 [ 2616.895898] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2616.899143] RIP: 0033:0x7fe02c11910b [ 2616.900709] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2616.912808] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2616.920087] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2616.927070] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2616.932535] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2616.935806] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2616.942259] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2644.057340] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2644.060096] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2644.064222] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2644.064362] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2644.064371] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2644.064382] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2644.064389] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) [ 2644.064374] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2644.064409] ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) [ 2644.064418] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2644.064422] libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 [ 2644.064431] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 [ 2644.064432] dns_resolver intel_rapl_msr [ 2644.064438] ORIG_RAX: ffffffffffffff13 [ 2644.064440] intel_rapl_common [ 2644.064444] RAX: 0000000000007b4c RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2644.064450] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2644.064452] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2644.064454] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2644.064457] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2644.064460] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2644.064463] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2644.064466] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2644.064475] Call Trace: [ 2644.064497] [ 2644.064507] ? show_regs.cold.9+0x22/0x2f [ 2644.064445] sb_edac rapl [ 2644.064518] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2644.064524] pcspkr i2c_piix4 squashfs ata_generic [ 2644.064532] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2644.064536] crct10dif_pclmul crc32_pclmul crc32c_intel [ 2644.064542] ? watchdog+0x40/0x40 [ 2644.064546] ghash_clmulni_intel ata_piix serio_raw [ 2644.064552] ? hrtimer_interrupt+0x128/0x2c0 [ 2644.064556] libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2644.064571] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2644.064563] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2644.064575] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2644.064586] ? apic_timer_interrupt+0xf/0x20 [ 2644.064592] [ 2644.064595] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2644.064601] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2644.064605] do_raw_spin_lock+0x177/0x190 [ 2644.064616] _raw_spin_lock+0x21/0x30 [ 2644.064583] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2644.064977] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2644.064995] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2644.064623] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2644.065185] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2644.065190] RAX: 0000000000007f21 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2644.065192] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2644.065194] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2644.065196] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2644.065198] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2644.065201] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2644.065203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2644.065205] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2644.065211] Call Trace: [ 2644.065213] [ 2644.065216] ? show_regs.cold.9+0x22/0x2f [ 2644.065226] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2644.065228] ? do_raw_spin_unlock+0x75/0x190 [ 2644.065234] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2644.065243] ? watchdog+0x40/0x40 [ 2644.065248] ? hrtimer_interrupt+0x128/0x2c0 [ 2644.065251] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2644.065257] ? apic_timer_interrupt+0xf/0x20 [ 2644.065261] [ 2644.065264] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2644.065268] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2644.065283] do_raw_spin_lock+0x177/0x190 [ 2644.065288] _raw_spin_lock+0x21/0x30 [ 2644.065242] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2644.065292] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2644.065731] process_one_work+0x2c8/0x700 [ 2644.065738] worker_thread+0x296/0x6e0 [ 2644.065742] ? rescuer_thread+0x570/0x570 [ 2644.065745] kthread+0x1d1/0x200 [ 2644.065750] ? set_kthread_struct+0x70/0x70 [ 2644.065753] ret_from_fork+0x1f/0x30 [ 2644.065895] ? do_wait_intr+0xf0/0xf0 [ 2644.065904] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2644.066119] mdc_reint+0x2d/0x110 [mdc] [ 2644.066161] mdc_unlink+0x658/0x870 [mdc] [ 2644.066188] lmv_unlink+0x34d/0xbd0 [lmv] [ 2644.066225] ll_unlink+0x24d/0x7f0 [lustre] [ 2644.066363] vfs_unlink+0x202/0x330 [ 2644.066372] do_unlinkat+0x2fa/0x460 [ 2644.066387] __x64_sys_unlink+0x37/0x50 [ 2644.066391] do_syscall_64+0xc1/0x440 [ 2644.066398] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2644.066404] RIP: 0033:0x7fe02c11910b [ 2644.066409] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2644.066413] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2644.066418] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2644.066421] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2644.066422] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2644.066424] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2644.066426] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2644.398890] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2644.420986] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2644.424352] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2644.428606] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2644.431607] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2644.448905] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2644.455727] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2644.463092] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2644.465455] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2644.468453] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2644.470847] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2644.472933] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2644.476213] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2644.478863] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2644.482298] Call Trace: [ 2644.483428] [ 2644.484341] ? show_regs.cold.9+0x22/0x2f [ 2644.486424] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2644.488767] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2644.490880] ? watchdog+0x40/0x40 [ 2644.492629] ? hrtimer_interrupt+0x128/0x2c0 [ 2644.494368] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2644.496920] ? apic_timer_interrupt+0xf/0x20 [ 2644.498920] [ 2644.500261] ? native_safe_halt+0x2e/0x50 [ 2644.502304] kvm_wait+0xcd/0xe0 [ 2644.503901] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2644.506371] do_raw_spin_lock+0x177/0x190 [ 2644.508534] _raw_spin_lock+0x21/0x30 [ 2644.510166] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2644.512760] obd_mod_free+0x46/0x140 [ptlrpc] [ 2644.515881] mdc_commit_open+0x91/0xb0 [mdc] [ 2644.517931] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2644.520816] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2644.523721] after_reply+0xe2c/0x1780 [ptlrpc] [ 2644.526285] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2644.528504] ? __next_timer_interrupt+0x160/0x160 [ 2644.530809] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2644.533411] ? do_wait_intr+0xf0/0xf0 [ 2644.535207] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2644.537835] kthread+0x1d1/0x200 [ 2644.539352] ? set_kthread_struct+0x70/0x70 [ 2644.541258] ret_from_fork+0x1f/0x30 [ 2672.059289] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2672.062467] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2672.064596] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) [ 2672.065482] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2672.065489] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2672.065569] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2672.065574] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2672.065580] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2672.065885] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2672.065896] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2672.065900] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2672.065905] RAX: 0000000000005e8a RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2672.065907] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2672.065910] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2672.065912] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2672.065914] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2672.065917] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2672.065920] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2672.065922] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2672.065931] Call Trace: [ 2672.065937] [ 2672.065942] ? show_regs.cold.9+0x22/0x2f [ 2672.065951] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2672.065958] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2672.065962] ? watchdog+0x40/0x40 [ 2672.065966] ? hrtimer_interrupt+0x128/0x2c0 [ 2672.065970] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2672.065978] ? apic_timer_interrupt+0xf/0x20 [ 2672.065982] [ 2672.065985] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2672.065989] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2672.065995] do_raw_spin_lock+0x177/0x190 [ 2672.066000] _raw_spin_lock+0x21/0x30 [ 2672.067597] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2672.067865] process_one_work+0x2c8/0x700 [ 2672.067874] worker_thread+0x296/0x6e0 [ 2672.067878] ? rescuer_thread+0x570/0x570 [ 2672.067882] kthread+0x1d1/0x200 [ 2672.067887] ? set_kthread_struct+0x70/0x70 [ 2672.067890] ret_from_fork+0x1f/0x30 [ 2672.072800] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2672.072892] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2672.072899] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2672.072902] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2672.072918] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2672.072923] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2672.072927] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2672.072929] RDX: 000000000000566f RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2672.072931] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2672.072933] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2672.072935] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2672.072939] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2672.072941] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2672.072943] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2672.072952] Call Trace: [ 2672.073098] [ 2672.092706] rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2672.092753] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2672.092760] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2672.092763] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2672.092777] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2672.092791] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2672.092795] RAX: 0000000000004ac7 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2672.092798] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2672.092800] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2672.092802] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2672.092804] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2672.092807] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2672.092810] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2672.092812] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2672.092819] Call Trace: [ 2672.092824] [ 2672.092829] ? show_regs.cold.9+0x22/0x2f [ 2672.092848] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2672.092856] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2672.092861] ? watchdog+0x40/0x40 [ 2672.092865] ? hrtimer_interrupt+0x128/0x2c0 [ 2672.092869] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2672.092877] ? apic_timer_interrupt+0xf/0x20 [ 2672.092882] [ 2672.092885] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2672.092890] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2672.092895] do_raw_spin_lock+0x177/0x190 [ 2672.092900] _raw_spin_lock+0x21/0x30 [ 2672.092906] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2672.100768] ? show_regs.cold.9+0x22/0x2f [ 2672.100804] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2672.100813] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2672.100818] ? watchdog+0x40/0x40 [ 2672.100822] ? hrtimer_interrupt+0x128/0x2c0 [ 2672.100827] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2672.100836] ? apic_timer_interrupt+0xf/0x20 [ 2672.100841] [ 2672.100846] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2672.100853] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2672.100858] do_raw_spin_lock+0x177/0x190 [ 2672.100864] _raw_spin_lock+0x21/0x30 [ 2672.101439] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2672.102067] obd_mod_free+0x46/0x140 [ptlrpc] [ 2672.102347] mdc_commit_open+0x91/0xb0 [mdc] [ 2672.102660] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2672.103081] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2672.104506] after_reply+0xe2c/0x1780 [ptlrpc] [ 2672.105733] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2672.106469] ? __next_timer_interrupt+0x160/0x160 [ 2672.106647] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2672.107104] ? do_wait_intr+0xf0/0xf0 [ 2672.107114] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2672.107393] kthread+0x1d1/0x200 [ 2672.107400] ? set_kthread_struct+0x70/0x70 [ 2672.107404] ret_from_fork+0x1f/0x30 [ 2672.996905] ? do_raw_spin_unlock+0x75/0x190 [ 2673.002662] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2673.009656] ? do_wait_intr+0xf0/0xf0 [ 2673.015924] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2673.026592] mdc_reint+0x2d/0x110 [mdc] [ 2673.034318] mdc_unlink+0x658/0x870 [mdc] [ 2673.038566] lmv_unlink+0x34d/0xbd0 [lmv] [ 2673.043592] ll_unlink+0x24d/0x7f0 [lustre] [ 2673.044947] vfs_unlink+0x202/0x330 [ 2673.051162] do_unlinkat+0x2fa/0x460 [ 2673.053989] __x64_sys_unlink+0x37/0x50 [ 2673.055969] do_syscall_64+0xc1/0x440 [ 2673.058163] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2673.060446] RIP: 0033:0x7fe02c11910b [ 2673.064717] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2673.086876] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2673.093118] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2673.100582] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2673.110958] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2673.118323] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2673.129558] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2700.057063] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2700.063696] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2700.064201] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2700.064208] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2700.064324] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2700.064329] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2700.064335] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2700.066281] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2700.066336] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2700.066345] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2700.066350] RAX: 0000000000006b4a RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2700.066352] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2700.066354] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2700.066356] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2700.066358] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2700.066361] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2700.066364] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2700.066366] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2700.066374] Call Trace: [ 2700.066393] [ 2700.066399] ? show_regs.cold.9+0x22/0x2f [ 2700.066465] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2700.066472] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2700.066476] ? watchdog+0x40/0x40 [ 2700.066480] ? hrtimer_interrupt+0x128/0x2c0 [ 2700.066484] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2700.066493] ? apic_timer_interrupt+0xf/0x20 [ 2700.066498] [ 2700.066502] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2700.066507] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2700.066512] do_raw_spin_lock+0x177/0x190 [ 2700.066517] _raw_spin_lock+0x21/0x30 [ 2700.066522] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2700.066907] process_one_work+0x2c8/0x700 [ 2700.066914] worker_thread+0x296/0x6e0 [ 2700.066918] ? rescuer_thread+0x570/0x570 [ 2700.066922] kthread+0x1d1/0x200 [ 2700.066927] ? set_kthread_struct+0x70/0x70 [ 2700.066931] ret_from_fork+0x1f/0x30 [ 2700.069671] Modules linked in: [ 2700.073616] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2700.073897] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2700.073903] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2700.073906] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2700.073919] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2700.073923] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2700.073928] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2700.073931] RDX: 000000000000179a RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2700.073933] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2700.073935] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2700.073938] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2700.073940] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2700.073943] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2700.073945] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2700.073952] Call Trace: [ 2700.073956] [ 2700.073961] ? show_regs.cold.9+0x22/0x2f [ 2700.083475] lustre(O) [ 2700.117646] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2700.117662] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2700.117665] ? watchdog+0x40/0x40 [ 2700.117668] ? hrtimer_interrupt+0x128/0x2c0 [ 2700.117670] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2700.117827] ? apic_timer_interrupt+0xf/0x20 [ 2700.117831] [ 2700.117835] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2700.117839] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2700.117842] do_raw_spin_lock+0x177/0x190 [ 2700.117847] _raw_spin_lock+0x21/0x30 [ 2700.117852] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2700.118067] obd_mod_free+0x46/0x140 [ptlrpc] [ 2700.118209] mdc_commit_open+0x91/0xb0 [mdc] [ 2700.118235] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2700.118495] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2700.118775] after_reply+0xe2c/0x1780 [ptlrpc] [ 2700.118972] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2700.132093] osp(O) [ 2700.137209] ? __next_timer_interrupt+0x160/0x160 [ 2700.137230] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2700.138217] ? do_wait_intr+0xf0/0xf0 [ 2700.140990] ofd(O) [ 2700.145969] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2700.146242] kthread+0x1d1/0x200 [ 2700.146251] ? set_kthread_struct+0x70/0x70 [ 2700.146267] ret_from_fork+0x1f/0x30 [ 2700.339773] lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2700.365366] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2700.372047] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2700.378387] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2700.383244] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2700.398779] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2700.406253] RAX: 0000000000000b51 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2700.411570] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2700.416356] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2700.420749] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2700.424861] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2700.431505] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2700.438432] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2700.441830] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2700.447945] Call Trace: [ 2700.448847] [ 2700.449593] ? show_regs.cold.9+0x22/0x2f [ 2700.455711] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2700.457494] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2700.461565] ? watchdog+0x40/0x40 [ 2700.464148] ? hrtimer_interrupt+0x128/0x2c0 [ 2700.466655] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2700.469541] ? apic_timer_interrupt+0xf/0x20 [ 2700.472192] [ 2700.473439] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2700.479166] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2700.484248] do_raw_spin_lock+0x177/0x190 [ 2700.487337] _raw_spin_lock+0x21/0x30 [ 2700.489473] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2700.497477] ? do_raw_spin_unlock+0x75/0x190 [ 2700.498988] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2700.504246] ? do_wait_intr+0xf0/0xf0 [ 2700.506941] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2700.510480] mdc_reint+0x2d/0x110 [mdc] [ 2700.514121] mdc_unlink+0x658/0x870 [mdc] [ 2700.518571] lmv_unlink+0x34d/0xbd0 [lmv] [ 2700.520623] ll_unlink+0x24d/0x7f0 [lustre] [ 2700.524491] vfs_unlink+0x202/0x330 [ 2700.531297] do_unlinkat+0x2fa/0x460 [ 2700.533643] __x64_sys_unlink+0x37/0x50 [ 2700.536548] do_syscall_64+0xc1/0x440 [ 2700.538875] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2700.541639] RIP: 0033:0x7fe02c11910b [ 2700.544298] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2700.556494] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2700.561783] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2700.566551] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2700.570302] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2700.574309] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2700.578142] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2728.057025] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2728.060618] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2728.060846] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2728.060916] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2728.060922] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2728.060925] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2728.060939] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2728.060943] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2728.060947] RAX: 0000000000005910 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2728.060950] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2728.060952] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2728.060955] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2728.060957] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2728.060960] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2728.060963] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2728.060965] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2728.060973] Call Trace: [ 2728.060979] [ 2728.060985] ? show_regs.cold.9+0x22/0x2f [ 2728.060996] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2728.061003] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2728.061007] ? watchdog+0x40/0x40 [ 2728.061013] ? hrtimer_interrupt+0x128/0x2c0 [ 2728.061017] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2728.061027] ? apic_timer_interrupt+0xf/0x20 [ 2728.061032] [ 2728.061036] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2728.061041] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2728.061046] do_raw_spin_lock+0x177/0x190 [ 2728.061051] _raw_spin_lock+0x21/0x30 [ 2728.061057] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2728.061344] ? do_raw_spin_unlock+0x75/0x190 [ 2728.061350] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2728.061546] ? do_wait_intr+0xf0/0xf0 [ 2728.061554] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2728.061748] mdc_reint+0x2d/0x110 [mdc] [ 2728.061790] mdc_unlink+0x658/0x870 [mdc] [ 2728.066505] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2728.066595] lmv_unlink+0x34d/0xbd0 [lmv] [ 2728.066633] ll_unlink+0x24d/0x7f0 [lustre] [ 2728.066752] vfs_unlink+0x202/0x330 [ 2728.066763] do_unlinkat+0x2fa/0x460 [ 2728.066769] __x64_sys_unlink+0x37/0x50 [ 2728.066773] do_syscall_64+0xc1/0x440 [ 2728.066779] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2728.066788] RIP: 0033:0x7fe02c11910b [ 2728.066796] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2728.066800] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2728.066815] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2728.066818] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2728.066824] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2728.066826] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2728.066828] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2728.069364] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2728.069438] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2728.069444] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2728.069447] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2728.069460] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2728.069464] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2728.069469] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2728.069471] RDX: 00000000000007e9 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2728.069474] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2728.069476] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2728.069478] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2728.069481] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2728.069484] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2728.069487] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2728.069495] Call Trace: [ 2728.069501] [ 2728.069506] ? show_regs.cold.9+0x22/0x2f [ 2728.069668] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2728.069674] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2728.069679] ? watchdog+0x40/0x40 [ 2728.069695] ? hrtimer_interrupt+0x128/0x2c0 [ 2728.069700] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2728.069707] ? apic_timer_interrupt+0xf/0x20 [ 2728.069711] [ 2728.069714] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2728.069720] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2728.069725] do_raw_spin_lock+0x177/0x190 [ 2728.069730] _raw_spin_lock+0x21/0x30 [ 2728.069735] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2728.070032] obd_mod_free+0x46/0x140 [ptlrpc] [ 2728.070367] mdc_commit_open+0x91/0xb0 [mdc] [ 2728.070408] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2728.070620] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2728.070846] after_reply+0xe2c/0x1780 [ptlrpc] [ 2728.071046] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2728.071386] ? __next_timer_interrupt+0x160/0x160 [ 2728.071398] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2728.071767] ? do_wait_intr+0xf0/0xf0 [ 2728.071778] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2728.072366] kthread+0x1d1/0x200 [ 2728.072373] ? set_kthread_struct+0x70/0x70 [ 2728.072377] ret_from_fork+0x1f/0x30 [ 2728.531725] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2728.548262] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2728.552095] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2728.555390] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2728.559058] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2728.562447] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2728.570777] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 2728.573830] RAX: 0000000000004189 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2728.576838] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2728.579659] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2728.583368] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2728.587099] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2728.589533] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2728.591624] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2728.593987] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2728.602379] Call Trace: [ 2728.604415] [ 2728.606401] ? show_regs.cold.9+0x22/0x2f [ 2728.608518] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2728.610459] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2728.613284] ? watchdog+0x40/0x40 [ 2728.615080] ? hrtimer_interrupt+0x128/0x2c0 [ 2728.616065] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2728.617398] ? apic_timer_interrupt+0xf/0x20 [ 2728.620417] [ 2728.620907] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2728.623857] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2728.626009] do_raw_spin_lock+0x177/0x190 [ 2728.629094] _raw_spin_lock+0x21/0x30 [ 2728.631153] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2728.633557] process_one_work+0x2c8/0x700 [ 2728.634999] worker_thread+0x296/0x6e0 [ 2728.636654] ? rescuer_thread+0x570/0x570 [ 2728.638478] kthread+0x1d1/0x200 [ 2728.642937] ? set_kthread_struct+0x70/0x70 [ 2728.647738] ret_from_fork+0x1f/0x30 [ 2742.112009] rcu: INFO: rcu_sched self-detected stall on CPU [ 2742.113968] rcu: 1-....: (385393 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=112704 [ 2742.120038] (t=600020 jiffies g=278549 q=4605) [ 2742.121887] Sending NMI from CPU 1 to CPUs 0: [ 2742.123920] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 [ 2742.124908] NMI backtrace for cpu 1 [ 2742.132049] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2742.136357] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2742.139405] Call Trace: [ 2742.140283] [ 2742.141143] ? dump_stack+0xbb/0x10e [ 2742.142355] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 2742.144100] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 2742.146417] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 2742.149195] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 2742.151287] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 2742.153164] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 2742.154900] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 2742.157307] ? update_process_times+0x6d/0xc0 [ 2742.159231] ? tick_sched_handle.isra.18+0x27/0x90 [ 2742.162143] ? tick_sched_timer+0x53/0xb0 [ 2742.164226] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2742.166548] ? tick_sched_handle.isra.18+0x90/0x90 [ 2742.169355] ? hrtimer_interrupt+0x128/0x2c0 [ 2742.171474] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2742.174097] ? apic_timer_interrupt+0xf/0x20 [ 2742.176291] [ 2742.177393] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2742.180077] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2742.182293] ? do_raw_spin_lock+0x177/0x190 [ 2742.184373] ? _raw_spin_lock+0x21/0x30 [ 2742.186017] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2742.188692] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2742.191126] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2742.192625] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2742.194822] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2742.197133] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2742.199180] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2742.201186] ? __next_timer_interrupt+0x160/0x160 [ 2742.202608] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2742.204484] ? do_wait_intr+0xf0/0xf0 [ 2742.205644] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2742.207539] ? kthread+0x1d1/0x200 [ 2742.208574] ? set_kthread_struct+0x70/0x70 [ 2742.209833] ? ret_from_fork+0x1f/0x30 [ 2756.064009] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2756.068946] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2756.094434] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2756.100125] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2756.104282] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2756.106997] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2756.109576] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2756.118708] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2756.121791] RAX: 0000000000003abe RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2756.124518] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2756.127773] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2756.131110] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2756.134494] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2756.137955] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2756.141992] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2756.144360] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2756.148020] Call Trace: [ 2756.149133] [ 2756.150075] ? show_regs.cold.9+0x22/0x2f [ 2756.152124] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2756.154627] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2756.157031] ? watchdog+0x40/0x40 [ 2756.158612] ? hrtimer_interrupt+0x128/0x2c0 [ 2756.160676] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2756.162954] ? apic_timer_interrupt+0xf/0x20 [ 2756.164990] [ 2756.166108] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2756.169211] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2756.172041] do_raw_spin_lock+0x177/0x190 [ 2756.173937] _raw_spin_lock+0x21/0x30 [ 2756.175630] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2756.178620] process_one_work+0x2c8/0x700 [ 2756.180068] worker_thread+0x296/0x6e0 [ 2756.181590] ? rescuer_thread+0x570/0x570 [ 2756.183837] kthread+0x1d1/0x200 [ 2756.185513] ? set_kthread_struct+0x70/0x70 [ 2756.187565] ret_from_fork+0x1f/0x30 [ 2768.057024] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2768.060017] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2768.065777] Modules linked in: [ 2768.069329] Modules linked in: [ 2768.069332] lustre(O) osp(O) [ 2768.070350] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2768.070424] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2768.070429] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2768.070432] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2768.070446] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2768.070450] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2768.070454] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2768.070456] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2768.070458] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2768.070460] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2768.070462] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2768.070465] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2768.070468] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2768.070470] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2768.070480] Call Trace: [ 2768.070528] [ 2768.070535] ? show_regs.cold.9+0x22/0x2f [ 2768.070546] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2768.070551] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2768.070556] ? watchdog+0x40/0x40 [ 2768.070560] ? hrtimer_interrupt+0x128/0x2c0 [ 2768.070564] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2768.070576] ? apic_timer_interrupt+0xf/0x20 [ 2768.070581] [ 2768.070584] ? native_safe_halt+0x2e/0x50 [ 2768.070588] kvm_wait+0xcd/0xe0 [ 2768.070595] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2768.070602] do_raw_spin_lock+0x177/0x190 [ 2768.070607] _raw_spin_lock+0x21/0x30 [ 2768.070611] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2768.070991] obd_mod_free+0x46/0x140 [ptlrpc] [ 2768.071280] mdc_commit_open+0x91/0xb0 [mdc] [ 2768.071329] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2768.071626] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2768.071897] after_reply+0xe2c/0x1780 [ptlrpc] [ 2768.072165] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2768.072363] ? __next_timer_interrupt+0x160/0x160 [ 2768.072372] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2768.072717] ? do_wait_intr+0xf0/0xf0 [ 2768.072726] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2768.072928] kthread+0x1d1/0x200 [ 2768.072934] ? set_kthread_struct+0x70/0x70 [ 2768.072937] ret_from_fork+0x1f/0x30 [ 2768.228740] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2768.242059] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2768.245164] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2768.247492] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2768.248720] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2768.254954] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2768.257259] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2768.259218] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 2768.261320] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2768.264340] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2768.268829] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2768.271981] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2768.275426] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2768.287246] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2768.292679] Call Trace: [ 2768.294603] [ 2768.295899] ? show_regs.cold.9+0x22/0x2f [ 2768.299390] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2768.300912] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2768.306922] ? watchdog+0x40/0x40 [ 2768.308870] ? hrtimer_interrupt+0x128/0x2c0 [ 2768.312635] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2768.315808] ? apic_timer_interrupt+0xf/0x20 [ 2768.316856] [ 2768.317515] ? native_safe_halt+0x2e/0x50 [ 2768.319029] kvm_wait+0xcd/0xe0 [ 2768.319862] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2768.321255] do_raw_spin_lock+0x177/0x190 [ 2768.324143] _raw_spin_lock+0x21/0x30 [ 2768.327971] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2768.338893] ? do_raw_spin_unlock+0x75/0x190 [ 2768.339929] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2768.347403] ? do_wait_intr+0xf0/0xf0 [ 2768.352776] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2768.359880] mdc_reint+0x2d/0x110 [mdc] [ 2768.361611] mdc_unlink+0x658/0x870 [mdc] [ 2768.363971] lmv_unlink+0x34d/0xbd0 [lmv] [ 2768.367794] ll_unlink+0x24d/0x7f0 [lustre] [ 2768.370142] vfs_unlink+0x202/0x330 [ 2768.371888] do_unlinkat+0x2fa/0x460 [ 2768.373381] __x64_sys_unlink+0x37/0x50 [ 2768.376869] do_syscall_64+0xc1/0x440 [ 2768.378910] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2768.381942] RIP: 0033:0x7fe02c11910b [ 2768.383913] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2768.401762] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2768.409233] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2768.415328] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2768.420405] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2768.424683] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2768.433282] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2784.066201] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2784.078774] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2784.152858] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2784.175393] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2784.188972] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2784.192191] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2784.199869] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2784.211334] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2784.219798] RAX: 0000000000004652 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2784.223114] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2784.225912] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2784.229925] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2784.233104] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2784.237740] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2784.241814] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2784.244753] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2784.249706] Call Trace: [ 2784.251148] [ 2784.252146] ? show_regs.cold.9+0x22/0x2f [ 2784.256295] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2784.263291] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2784.264795] ? watchdog+0x40/0x40 [ 2784.268493] ? hrtimer_interrupt+0x128/0x2c0 [ 2784.272622] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2784.274531] ? apic_timer_interrupt+0xf/0x20 [ 2784.278613] [ 2784.279332] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2784.285415] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2784.288167] do_raw_spin_lock+0x177/0x190 [ 2784.290287] _raw_spin_lock+0x21/0x30 [ 2784.291840] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2784.294698] process_one_work+0x2c8/0x700 [ 2784.296767] worker_thread+0x296/0x6e0 [ 2784.298490] ? rescuer_thread+0x570/0x570 [ 2784.300186] kthread+0x1d1/0x200 [ 2784.301762] ? set_kthread_struct+0x70/0x70 [ 2784.303702] ret_from_fork+0x1f/0x30 [ 2796.057055] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2796.060484] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2796.062878] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2796.062959] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2796.062966] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2796.062968] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2796.062981] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2796.062986] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2796.062990] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2796.062992] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 2796.063004] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2796.063007] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2796.063009] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2796.063013] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2796.063015] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2796.063018] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2796.063025] Call Trace: [ 2796.063030] [ 2796.063035] ? show_regs.cold.9+0x22/0x2f [ 2796.063045] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2796.063053] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2796.063057] ? watchdog+0x40/0x40 [ 2796.063062] ? hrtimer_interrupt+0x128/0x2c0 [ 2796.063066] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2796.063072] ? apic_timer_interrupt+0xf/0x20 [ 2796.063076] [ 2796.063079] ? native_safe_halt+0x2e/0x50 [ 2796.063082] kvm_wait+0xcd/0xe0 [ 2796.063089] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2796.063096] do_raw_spin_lock+0x177/0x190 [ 2796.063100] _raw_spin_lock+0x21/0x30 [ 2796.063104] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2796.068838] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2796.068917] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2796.068925] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2796.068928] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2796.068942] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2796.068946] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2796.068950] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2796.068952] RDX: 000000000000321c RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2796.068954] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2796.068956] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2796.068958] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2796.068961] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2796.068964] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2796.068967] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2796.068975] Call Trace: [ 2796.068980] [ 2796.068986] ? show_regs.cold.9+0x22/0x2f [ 2796.068994] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2796.069000] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2796.069004] ? watchdog+0x40/0x40 [ 2796.069009] ? hrtimer_interrupt+0x128/0x2c0 [ 2796.069013] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2796.069021] ? apic_timer_interrupt+0xf/0x20 [ 2796.069026] [ 2796.069029] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2796.069033] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2796.069038] do_raw_spin_lock+0x177/0x190 [ 2796.069042] _raw_spin_lock+0x21/0x30 [ 2796.069047] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2796.069319] obd_mod_free+0x46/0x140 [ptlrpc] [ 2796.069520] mdc_commit_open+0x91/0xb0 [mdc] [ 2796.069558] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2796.069762] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2796.069961] after_reply+0xe2c/0x1780 [ptlrpc] [ 2796.070176] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2796.070383] ? __next_timer_interrupt+0x160/0x160 [ 2796.070392] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2796.070591] ? do_wait_intr+0xf0/0xf0 [ 2796.070601] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2796.111365] ? do_raw_spin_unlock+0x75/0x190 [ 2796.111387] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2796.112112] ? do_wait_intr+0xf0/0xf0 [ 2796.112121] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2796.112456] mdc_reint+0x2d/0x110 [mdc] [ 2796.112496] mdc_unlink+0x658/0x870 [mdc] [ 2796.112527] lmv_unlink+0x34d/0xbd0 [lmv] [ 2796.112555] ll_unlink+0x24d/0x7f0 [lustre] [ 2796.112756] vfs_unlink+0x202/0x330 [ 2796.112764] do_unlinkat+0x2fa/0x460 [ 2796.112769] __x64_sys_unlink+0x37/0x50 [ 2796.112772] do_syscall_64+0xc1/0x440 [ 2796.112778] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2796.112786] RIP: 0033:0x7fe02c11910b [ 2796.112794] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2796.112798] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2796.112803] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2796.112805] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2796.112807] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2796.112809] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2796.112811] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2796.498548] kthread+0x1d1/0x200 [ 2796.501256] ? set_kthread_struct+0x70/0x70 [ 2796.504796] ret_from_fork+0x1f/0x30 [ 2812.064976] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2812.068652] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2812.121403] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2812.133637] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2812.142062] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2812.148265] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2812.150310] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2812.173075] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2812.182679] RAX: 000000000000758d RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2812.193760] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2812.203903] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2812.210450] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2812.216317] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2812.223643] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2812.229988] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2812.241079] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2812.246646] Call Trace: [ 2812.247674] [ 2812.253164] ? show_regs.cold.9+0x22/0x2f [ 2812.257577] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2812.264566] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2812.266468] ? watchdog+0x40/0x40 [ 2812.274340] ? hrtimer_interrupt+0x128/0x2c0 [ 2812.275880] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2812.282733] ? apic_timer_interrupt+0xf/0x20 [ 2812.284231] [ 2812.289256] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2812.295070] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2812.300376] do_raw_spin_lock+0x177/0x190 [ 2812.309791] _raw_spin_lock+0x21/0x30 [ 2812.311040] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2812.315563] process_one_work+0x2c8/0x700 [ 2812.322124] worker_thread+0x296/0x6e0 [ 2812.325389] ? rescuer_thread+0x570/0x570 [ 2812.334657] kthread+0x1d1/0x200 [ 2812.335771] ? set_kthread_struct+0x70/0x70 [ 2812.341125] ret_from_fork+0x1f/0x30 [ 2824.057332] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2824.060010] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2824.062548] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2824.062620] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2824.062627] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2824.062630] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2824.062644] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2824.062649] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2824.062653] RAX: 0000000000007604 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2824.062656] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2824.062658] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2824.062660] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2824.062663] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2824.062666] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2824.062669] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2824.062671] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2824.062680] Call Trace: [ 2824.062685] [ 2824.062691] ? show_regs.cold.9+0x22/0x2f [ 2824.062701] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2824.062708] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2824.062711] ? watchdog+0x40/0x40 [ 2824.062715] ? hrtimer_interrupt+0x128/0x2c0 [ 2824.062720] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2824.062728] ? apic_timer_interrupt+0xf/0x20 [ 2824.062733] [ 2824.062736] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2824.062741] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2824.062745] do_raw_spin_lock+0x177/0x190 [ 2824.062750] _raw_spin_lock+0x21/0x30 [ 2824.062755] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2824.063079] ? do_raw_spin_unlock+0x75/0x190 [ 2824.063082] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2824.072837] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2824.073084] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2824.106114] ? do_wait_intr+0xf0/0xf0 [ 2824.106145] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2824.106402] mdc_reint+0x2d/0x110 [mdc] [ 2824.106439] mdc_unlink+0x658/0x870 [mdc] [ 2824.106466] lmv_unlink+0x34d/0xbd0 [lmv] [ 2824.106496] ll_unlink+0x24d/0x7f0 [lustre] [ 2824.106611] vfs_unlink+0x202/0x330 [ 2824.106619] do_unlinkat+0x2fa/0x460 [ 2824.106625] __x64_sys_unlink+0x37/0x50 [ 2824.106628] do_syscall_64+0xc1/0x440 [ 2824.106635] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2824.106642] RIP: 0033:0x7fe02c11910b [ 2824.106650] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2824.106654] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2824.106659] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2824.106661] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2824.106664] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2824.106666] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2824.106668] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2824.306292] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2824.315186] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2824.318752] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2824.329966] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2824.334588] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2824.339320] RDX: 000000000000670b RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2824.342436] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2824.346392] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2824.350184] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2824.353771] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2824.358322] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2824.361666] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2824.365468] Call Trace: [ 2824.368026] [ 2824.368886] ? show_regs.cold.9+0x22/0x2f [ 2824.370493] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2824.372812] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2824.374717] ? watchdog+0x40/0x40 [ 2824.376236] ? hrtimer_interrupt+0x128/0x2c0 [ 2824.378403] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2824.382170] ? apic_timer_interrupt+0xf/0x20 [ 2824.383557] [ 2824.384383] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2824.386203] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2824.391555] do_raw_spin_lock+0x177/0x190 [ 2824.394469] _raw_spin_lock+0x21/0x30 [ 2824.396594] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2824.398586] obd_mod_free+0x46/0x140 [ptlrpc] [ 2824.407296] mdc_commit_open+0x91/0xb0 [mdc] [ 2824.410229] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2824.418824] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2824.427238] after_reply+0xe2c/0x1780 [ptlrpc] [ 2824.434338] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2824.441405] ? __next_timer_interrupt+0x160/0x160 [ 2824.450102] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2824.462039] ? do_wait_intr+0xf0/0xf0 [ 2824.463479] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2824.475240] kthread+0x1d1/0x200 [ 2824.476374] ? set_kthread_struct+0x70/0x70 [ 2824.477955] ret_from_fork+0x1f/0x30 [ 2840.064497] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2840.068195] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2840.139914] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2840.163557] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2840.172740] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2840.185632] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2840.189009] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2840.205138] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2840.208008] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2840.210670] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2840.212613] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2840.215826] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2840.218927] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2840.225202] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2840.228239] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2840.234404] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2840.239175] Call Trace: [ 2840.240024] [ 2840.240903] ? show_regs.cold.9+0x22/0x2f [ 2840.242427] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2840.244067] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2840.246182] ? watchdog+0x40/0x40 [ 2840.247968] ? hrtimer_interrupt+0x128/0x2c0 [ 2840.249365] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2840.251430] ? apic_timer_interrupt+0xf/0x20 [ 2840.256865] [ 2840.257782] ? native_safe_halt+0x2e/0x50 [ 2840.259418] kvm_wait+0xcd/0xe0 [ 2840.261596] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2840.263674] do_raw_spin_lock+0x177/0x190 [ 2840.269651] _raw_spin_lock+0x21/0x30 [ 2840.270823] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2840.280035] process_one_work+0x2c8/0x700 [ 2840.284559] worker_thread+0x296/0x6e0 [ 2840.286735] ? rescuer_thread+0x570/0x570 [ 2840.288556] kthread+0x1d1/0x200 [ 2840.290163] ? set_kthread_struct+0x70/0x70 [ 2840.292683] ret_from_fork+0x1f/0x30 [ 2852.059020] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2852.060089] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2852.066550] Modules linked in: lustre(O) [ 2852.070449] Modules linked in: [ 2852.070451] osp(O) ofd(O) lod(O) [ 2852.072966] lustre(O) [ 2852.074235] mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common [ 2852.075674] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr [ 2852.077371] sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2852.077495] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2852.089538] i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2852.089575] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2852.089582] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2852.089585] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2852.089599] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2852.089603] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2852.089608] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2852.089611] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2852.089612] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2852.089615] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2852.089617] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2852.089620] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2852.089623] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2852.089625] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2852.089641] Call Trace: [ 2852.089648] [ 2852.089653] ? show_regs.cold.9+0x22/0x2f [ 2852.089664] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2852.089671] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2852.089675] ? watchdog+0x40/0x40 [ 2852.089680] ? hrtimer_interrupt+0x128/0x2c0 [ 2852.089683] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2852.089691] ? apic_timer_interrupt+0xf/0x20 [ 2852.089695] [ 2852.089697] ? native_safe_halt+0x2e/0x50 [ 2852.089701] kvm_wait+0xcd/0xe0 [ 2852.089708] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2852.089715] do_raw_spin_lock+0x177/0x190 [ 2852.089719] _raw_spin_lock+0x21/0x30 [ 2852.089725] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2852.090328] obd_mod_free+0x46/0x140 [ptlrpc] [ 2852.090594] mdc_commit_open+0x91/0xb0 [mdc] [ 2852.090663] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2852.091125] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2852.124201] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2852.136671] after_reply+0xe2c/0x1780 [ptlrpc] [ 2852.141777] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2852.148512] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2852.155739] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2852.155750] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2852.155757] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2852.155759] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 2852.155761] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2852.155764] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2852.155766] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2852.155769] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2852.155772] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2852.155774] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2852.155781] Call Trace: [ 2852.155786] [ 2852.155791] ? show_regs.cold.9+0x22/0x2f [ 2852.155801] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2852.155807] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2852.155811] ? watchdog+0x40/0x40 [ 2852.155815] ? hrtimer_interrupt+0x128/0x2c0 [ 2852.155819] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2852.155827] ? apic_timer_interrupt+0xf/0x20 [ 2852.155831] [ 2852.155833] ? native_safe_halt+0x2e/0x50 [ 2852.155838] kvm_wait+0xcd/0xe0 [ 2852.155844] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2852.155851] do_raw_spin_lock+0x177/0x190 [ 2852.155854] _raw_spin_lock+0x21/0x30 [ 2852.155860] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2852.159548] ? __next_timer_interrupt+0x160/0x160 [ 2852.160844] ? do_raw_spin_unlock+0x75/0x190 [ 2852.160857] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2852.170186] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2852.173336] ? do_wait_intr+0xf0/0xf0 [ 2852.173353] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2852.175639] ? do_wait_intr+0xf0/0xf0 [ 2852.180956] mdc_reint+0x2d/0x110 [mdc] [ 2852.184063] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2852.184421] kthread+0x1d1/0x200 [ 2852.184430] ? set_kthread_struct+0x70/0x70 [ 2852.184434] ret_from_fork+0x1f/0x30 [ 2852.349253] mdc_unlink+0x658/0x870 [mdc] [ 2852.350556] lmv_unlink+0x34d/0xbd0 [lmv] [ 2852.351813] ll_unlink+0x24d/0x7f0 [lustre] [ 2852.353559] vfs_unlink+0x202/0x330 [ 2852.354664] do_unlinkat+0x2fa/0x460 [ 2852.356543] __x64_sys_unlink+0x37/0x50 [ 2852.357834] do_syscall_64+0xc1/0x440 [ 2852.359070] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2852.361675] RIP: 0033:0x7fe02c11910b [ 2852.362817] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2852.369413] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2852.371996] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2852.375282] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2852.378861] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2852.381316] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2852.383808] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2868.064026] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2868.068826] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2868.104611] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2868.113779] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2868.117658] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2868.123369] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2868.125419] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2868.134955] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 2868.140449] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2868.149027] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2868.154228] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2868.158055] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2868.162932] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2868.171182] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2868.177025] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2868.179684] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2868.185079] Call Trace: [ 2868.185858] [ 2868.188426] ? show_regs.cold.9+0x22/0x2f [ 2868.192296] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2868.196121] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2868.197766] ? watchdog+0x40/0x40 [ 2868.200463] ? hrtimer_interrupt+0x128/0x2c0 [ 2868.206015] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2868.209817] ? apic_timer_interrupt+0xf/0x20 [ 2868.214287] [ 2868.215060] ? native_safe_halt+0x2e/0x50 [ 2868.216554] kvm_wait+0xcd/0xe0 [ 2868.220488] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2868.224251] do_raw_spin_lock+0x177/0x190 [ 2868.226389] _raw_spin_lock+0x21/0x30 [ 2868.228098] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2868.230707] process_one_work+0x2c8/0x700 [ 2868.232176] worker_thread+0x296/0x6e0 [ 2868.236281] ? rescuer_thread+0x570/0x570 [ 2868.239233] kthread+0x1d1/0x200 [ 2868.245636] ? set_kthread_struct+0x70/0x70 [ 2868.247472] ret_from_fork+0x1f/0x30 [ 2880.057939] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2880.065231] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2880.066479] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2880.066819] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2880.066826] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2880.066829] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2880.066842] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2880.066846] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2880.066850] RAX: 0000000000002b23 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2880.066853] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2880.066855] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2880.066857] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2880.066860] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2880.066863] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2880.066866] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2880.066868] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2880.066875] Call Trace: [ 2880.066880] [ 2880.066885] ? show_regs.cold.9+0x22/0x2f [ 2880.066893] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2880.066898] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2880.066902] ? watchdog+0x40/0x40 [ 2880.066907] ? hrtimer_interrupt+0x128/0x2c0 [ 2880.066911] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2880.066918] ? apic_timer_interrupt+0xf/0x20 [ 2880.066922] [ 2880.066925] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2880.066929] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2880.066934] do_raw_spin_lock+0x177/0x190 [ 2880.066938] _raw_spin_lock+0x21/0x30 [ 2880.066943] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2880.067222] ? do_raw_spin_unlock+0x75/0x190 [ 2880.067227] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2880.075278] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2880.075352] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2880.075359] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2880.075362] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2880.075375] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2880.075379] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2880.075384] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2880.075386] RDX: 0000000000007402 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2880.075388] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2880.075390] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2880.075392] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2880.075396] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2880.075408] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2880.075410] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2880.075419] Call Trace: [ 2880.075424] [ 2880.075429] ? show_regs.cold.9+0x22/0x2f [ 2880.075437] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2880.075443] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2880.075447] ? watchdog+0x40/0x40 [ 2880.075451] ? hrtimer_interrupt+0x128/0x2c0 [ 2880.075455] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2880.075462] ? apic_timer_interrupt+0xf/0x20 [ 2880.075467] [ 2880.075469] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2880.075473] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2880.075478] do_raw_spin_lock+0x177/0x190 [ 2880.075482] _raw_spin_lock+0x21/0x30 [ 2880.075487] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2880.076386] obd_mod_free+0x46/0x140 [ptlrpc] [ 2880.076510] mdc_commit_open+0x91/0xb0 [mdc] [ 2880.076560] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2880.076753] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2880.123041] ? do_wait_intr+0xf0/0xf0 [ 2880.123062] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2880.143387] after_reply+0xe2c/0x1780 [ptlrpc] [ 2880.144135] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2880.144883] ? __next_timer_interrupt+0x160/0x160 [ 2880.144894] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2880.145118] ? do_wait_intr+0xf0/0xf0 [ 2880.145128] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2880.145980] kthread+0x1d1/0x200 [ 2880.145987] ? set_kthread_struct+0x70/0x70 [ 2880.145992] ret_from_fork+0x1f/0x30 [ 2880.603875] mdc_reint+0x2d/0x110 [mdc] [ 2880.605352] mdc_unlink+0x658/0x870 [mdc] [ 2880.606946] lmv_unlink+0x34d/0xbd0 [lmv] [ 2880.608441] ll_unlink+0x24d/0x7f0 [lustre] [ 2880.610317] vfs_unlink+0x202/0x330 [ 2880.611752] do_unlinkat+0x2fa/0x460 [ 2880.612994] __x64_sys_unlink+0x37/0x50 [ 2880.618705] do_syscall_64+0xc1/0x440 [ 2880.620174] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2880.622325] RIP: 0033:0x7fe02c11910b [ 2880.628432] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2880.639292] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2880.646368] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2880.649746] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2880.652929] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2880.655831] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2880.672409] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2896.064132] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2896.071872] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2896.100461] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2896.107204] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2896.115569] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2896.121150] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2896.125420] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2896.137747] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2896.142026] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2896.145808] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2896.147751] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2896.151809] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2896.156404] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2896.160923] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2896.167797] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2896.171076] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2896.175692] Call Trace: [ 2896.176668] [ 2896.177623] ? show_regs.cold.9+0x22/0x2f [ 2896.181060] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2896.184594] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2896.186804] ? watchdog+0x40/0x40 [ 2896.188694] ? hrtimer_interrupt+0x128/0x2c0 [ 2896.190716] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2896.193143] ? apic_timer_interrupt+0xf/0x20 [ 2896.195104] [ 2896.196359] ? native_safe_halt+0x2e/0x50 [ 2896.199158] kvm_wait+0xcd/0xe0 [ 2896.200792] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2896.204142] do_raw_spin_lock+0x177/0x190 [ 2896.206597] _raw_spin_lock+0x21/0x30 [ 2896.208524] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2896.211779] process_one_work+0x2c8/0x700 [ 2896.215405] worker_thread+0x296/0x6e0 [ 2896.216979] ? rescuer_thread+0x570/0x570 [ 2896.219178] kthread+0x1d1/0x200 [ 2896.222774] ? set_kthread_struct+0x70/0x70 [ 2896.226035] ret_from_fork+0x1f/0x30 [ 2908.058856] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 2908.060016] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 2908.071787] Modules linked in: lustre(O) osp(O) ofd(O) [ 2908.074532] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2908.074620] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2908.074630] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2908.074633] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2908.074647] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2908.074662] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2908.074666] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2908.074668] RDX: 0000000000005131 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2908.074670] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2908.074672] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2908.074674] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2908.074677] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2908.074680] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2908.074682] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2908.074693] Call Trace: [ 2908.074698] [ 2908.074703] ? show_regs.cold.9+0x22/0x2f [ 2908.074712] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2908.074718] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2908.074722] ? watchdog+0x40/0x40 [ 2908.074727] ? hrtimer_interrupt+0x128/0x2c0 [ 2908.074732] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2908.074744] ? apic_timer_interrupt+0xf/0x20 [ 2908.074749] [ 2908.074752] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2908.074756] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2908.074761] do_raw_spin_lock+0x177/0x190 [ 2908.074766] _raw_spin_lock+0x21/0x30 [ 2908.074772] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2908.075099] obd_mod_free+0x46/0x140 [ptlrpc] [ 2908.075415] mdc_commit_open+0x91/0xb0 [mdc] [ 2908.075450] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2908.075692] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2908.075932] after_reply+0xe2c/0x1780 [ptlrpc] [ 2908.076231] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2908.076532] ? __next_timer_interrupt+0x160/0x160 [ 2908.076542] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2908.076799] ? do_wait_intr+0xf0/0xf0 [ 2908.076808] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2908.077003] kthread+0x1d1/0x200 [ 2908.077009] ? set_kthread_struct+0x70/0x70 [ 2908.077013] ret_from_fork+0x1f/0x30 [ 2908.241253] lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2908.258023] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2908.263593] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2908.267911] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2908.271169] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2908.280139] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2908.283843] RAX: 000000000000792f RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2908.287162] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2908.290386] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2908.293259] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2908.296044] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2908.300050] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2908.305205] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2908.316544] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2908.322314] Call Trace: [ 2908.323175] [ 2908.324003] ? show_regs.cold.9+0x22/0x2f [ 2908.327669] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2908.330223] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2908.332500] ? watchdog+0x40/0x40 [ 2908.334337] ? hrtimer_interrupt+0x128/0x2c0 [ 2908.339974] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2908.342378] ? apic_timer_interrupt+0xf/0x20 [ 2908.344147] [ 2908.344848] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2908.346785] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2908.348603] do_raw_spin_lock+0x177/0x190 [ 2908.350142] _raw_spin_lock+0x21/0x30 [ 2908.351212] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2908.353159] ? do_raw_spin_unlock+0x75/0x190 [ 2908.354683] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2908.356535] ? do_wait_intr+0xf0/0xf0 [ 2908.357837] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2908.359712] mdc_reint+0x2d/0x110 [mdc] [ 2908.361083] mdc_unlink+0x658/0x870 [mdc] [ 2908.362291] lmv_unlink+0x34d/0xbd0 [lmv] [ 2908.363674] ll_unlink+0x24d/0x7f0 [lustre] [ 2908.365672] vfs_unlink+0x202/0x330 [ 2908.367205] do_unlinkat+0x2fa/0x460 [ 2908.368217] __x64_sys_unlink+0x37/0x50 [ 2908.369349] do_syscall_64+0xc1/0x440 [ 2908.371577] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2908.373996] RIP: 0033:0x7fe02c11910b [ 2908.376238] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2908.393208] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2908.396786] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2908.400326] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2908.403632] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2908.406574] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2908.410025] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2922.115668] rcu: INFO: rcu_sched self-detected stall on CPU [ 2922.118032] rcu: 1-....: (514040 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=148573 [ 2922.131647] (t=780023 jiffies g=278549 q=5382) [ 2922.134290] Sending NMI from CPU 1 to CPUs 0: [ 2922.138738] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 3.136 msecs [ 2922.141324] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 [ 2922.141334] NMI backtrace for cpu 1 [ 2922.157536] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2922.164281] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2922.169625] Call Trace: [ 2922.170960] [ 2922.171642] ? dump_stack+0xbb/0x10e [ 2922.172977] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 2922.175132] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 2922.176660] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 2922.178564] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 2922.180690] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 2922.182661] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 2922.184484] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 2922.186476] ? update_process_times+0x6d/0xc0 [ 2922.187906] ? tick_sched_handle.isra.18+0x27/0x90 [ 2922.190364] ? tick_sched_timer+0x53/0xb0 [ 2922.191874] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2922.193891] ? tick_sched_handle.isra.18+0x90/0x90 [ 2922.195934] ? hrtimer_interrupt+0x128/0x2c0 [ 2922.198181] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2922.200235] ? apic_timer_interrupt+0xf/0x20 [ 2922.202310] [ 2922.203191] ? native_safe_halt+0x2e/0x50 [ 2922.204959] ? kvm_wait+0xcd/0xe0 [ 2922.206161] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2922.208031] ? do_raw_spin_lock+0x177/0x190 [ 2922.209968] ? _raw_spin_lock+0x21/0x30 [ 2922.211913] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2922.214371] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2922.216477] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2922.217848] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2922.219786] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2922.223288] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2922.225743] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2922.228308] ? __next_timer_interrupt+0x160/0x160 [ 2922.231113] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2922.235141] ? do_wait_intr+0xf0/0xf0 [ 2922.236612] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2922.238538] ? kthread+0x1d1/0x200 [ 2922.250197] ? set_kthread_struct+0x70/0x70 [ 2922.254473] ? ret_from_fork+0x1f/0x30 [ 2924.064269] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2924.068750] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2924.111248] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2924.123418] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2924.133988] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2924.140857] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2924.146001] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2924.163476] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2924.170532] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2924.184863] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2924.189654] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2924.193254] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2924.198340] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2924.203602] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2924.209931] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2924.213401] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2924.217471] Call Trace: [ 2924.218935] [ 2924.220327] ? show_regs.cold.9+0x22/0x2f [ 2924.222893] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2924.228347] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2924.229911] ? watchdog+0x40/0x40 [ 2924.237519] ? hrtimer_interrupt+0x128/0x2c0 [ 2924.247494] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2924.254352] ? apic_timer_interrupt+0xf/0x20 [ 2924.258938] [ 2924.261491] ? native_safe_halt+0x2e/0x50 [ 2924.264005] kvm_wait+0xcd/0xe0 [ 2924.266506] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2924.269201] do_raw_spin_lock+0x177/0x190 [ 2924.272360] _raw_spin_lock+0x21/0x30 [ 2924.273695] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2924.283293] process_one_work+0x2c8/0x700 [ 2924.286922] worker_thread+0x296/0x6e0 [ 2924.290625] ? rescuer_thread+0x570/0x570 [ 2924.302292] kthread+0x1d1/0x200 [ 2924.309486] ? set_kthread_struct+0x70/0x70 [ 2924.316482] ret_from_fork+0x1f/0x30 [ 2948.059627] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2948.064652] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2948.068799] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2948.068876] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2948.068884] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2948.068887] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2948.068902] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2948.068907] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2948.068911] RAX: 0000000000007f90 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2948.068914] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2948.068916] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2948.068917] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2948.068920] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2948.068923] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2948.068925] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2948.068927] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2948.068937] Call Trace: [ 2948.068943] [ 2948.068949] ? show_regs.cold.9+0x22/0x2f [ 2948.074037] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2948.074116] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2948.074123] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2948.074126] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2948.074139] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2948.074143] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2948.074147] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2948.074150] RDX: 0000000000007470 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2948.074152] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2948.074154] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2948.074156] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2948.074159] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2948.074162] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2948.074164] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2948.074173] Call Trace: [ 2948.074178] [ 2948.074185] ? show_regs.cold.9+0x22/0x2f [ 2948.074193] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2948.074200] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2948.074204] ? watchdog+0x40/0x40 [ 2948.074209] ? hrtimer_interrupt+0x128/0x2c0 [ 2948.074213] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2948.074222] ? apic_timer_interrupt+0xf/0x20 [ 2948.074227] [ 2948.074230] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2948.074235] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2948.074239] do_raw_spin_lock+0x177/0x190 [ 2948.074244] _raw_spin_lock+0x21/0x30 [ 2948.074250] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2948.074514] obd_mod_free+0x46/0x140 [ptlrpc] [ 2948.106315] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2948.121692] mdc_commit_open+0x91/0xb0 [mdc] [ 2948.122824] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2948.130421] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2948.130438] ? watchdog+0x40/0x40 [ 2948.130444] ? hrtimer_interrupt+0x128/0x2c0 [ 2948.130447] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2948.130455] ? apic_timer_interrupt+0xf/0x20 [ 2948.130460] [ 2948.130464] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2948.130470] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2948.130475] do_raw_spin_lock+0x177/0x190 [ 2948.130481] _raw_spin_lock+0x21/0x30 [ 2948.130487] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2948.130760] ? do_raw_spin_unlock+0x75/0x190 [ 2948.130766] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2948.130977] ? do_wait_intr+0xf0/0xf0 [ 2948.130985] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2948.131185] mdc_reint+0x2d/0x110 [mdc] [ 2948.131220] mdc_unlink+0x658/0x870 [mdc] [ 2948.131245] lmv_unlink+0x34d/0xbd0 [lmv] [ 2948.131271] ll_unlink+0x24d/0x7f0 [lustre] [ 2948.131387] vfs_unlink+0x202/0x330 [ 2948.131394] do_unlinkat+0x2fa/0x460 [ 2948.131399] __x64_sys_unlink+0x37/0x50 [ 2948.131403] do_syscall_64+0xc1/0x440 [ 2948.131409] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2948.131417] RIP: 0033:0x7fe02c11910b [ 2948.131425] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2948.131429] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2948.131434] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2948.131436] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2948.131438] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2948.131440] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2948.131443] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2948.782900] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2948.787146] after_reply+0xe2c/0x1780 [ptlrpc] [ 2948.795216] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2948.797307] ? __next_timer_interrupt+0x160/0x160 [ 2948.806546] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2948.816063] ? do_wait_intr+0xf0/0xf0 [ 2948.821517] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2948.827275] kthread+0x1d1/0x200 [ 2948.831703] ? set_kthread_struct+0x70/0x70 [ 2948.835850] ret_from_fork+0x1f/0x30 [ 2952.065663] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 2952.074663] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2952.138470] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2952.147315] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2952.164807] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2952.176718] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2952.188320] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2952.211416] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2952.219777] RAX: 0000000000007f84 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2952.226648] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2952.235802] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2952.241968] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2952.249738] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2952.261793] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2952.268696] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2952.274760] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2952.284310] Call Trace: [ 2952.285939] [ 2952.286774] ? show_regs.cold.9+0x22/0x2f [ 2952.289991] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2952.291530] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2952.295612] ? watchdog+0x40/0x40 [ 2952.297174] ? hrtimer_interrupt+0x128/0x2c0 [ 2952.307339] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2952.310870] ? apic_timer_interrupt+0xf/0x20 [ 2952.318443] [ 2952.319422] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2952.324097] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2952.330803] do_raw_spin_lock+0x177/0x190 [ 2952.336939] _raw_spin_lock+0x21/0x30 [ 2952.338225] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2952.342670] process_one_work+0x2c8/0x700 [ 2952.344859] worker_thread+0x296/0x6e0 [ 2952.347075] ? rescuer_thread+0x570/0x570 [ 2952.348316] kthread+0x1d1/0x200 [ 2952.353680] ? set_kthread_struct+0x70/0x70 [ 2952.357076] ret_from_fork+0x1f/0x30 [ 2976.058868] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 2976.060009] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 2976.064835] Modules linked in: [ 2976.067331] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2976.067411] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2976.067418] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2976.067421] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2976.067435] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2976.067439] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2976.067443] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 2976.067446] RDX: 0000000000004943 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 2976.067448] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 2976.067450] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 2976.067452] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2976.067455] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 2976.067458] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2976.067472] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 2976.067480] Call Trace: [ 2976.067485] [ 2976.067491] ? show_regs.cold.9+0x22/0x2f [ 2976.067500] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2976.067506] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2976.067510] ? watchdog+0x40/0x40 [ 2976.067514] ? hrtimer_interrupt+0x128/0x2c0 [ 2976.067518] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2976.067526] ? apic_timer_interrupt+0xf/0x20 [ 2976.067531] [ 2976.067533] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2976.067538] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2976.067543] do_raw_spin_lock+0x177/0x190 [ 2976.067547] _raw_spin_lock+0x21/0x30 [ 2976.067552] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2976.069702] lustre(O) [ 2976.092589] obd_mod_free+0x46/0x140 [ptlrpc] [ 2976.093026] mdc_commit_open+0x91/0xb0 [mdc] [ 2976.093071] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2976.093383] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2976.093590] after_reply+0xe2c/0x1780 [ptlrpc] [ 2976.093864] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2976.094066] ? __next_timer_interrupt+0x160/0x160 [ 2976.094078] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2976.094398] ? do_wait_intr+0xf0/0xf0 [ 2976.094408] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2976.094653] kthread+0x1d1/0x200 [ 2976.094659] ? set_kthread_struct+0x70/0x70 [ 2976.094663] ret_from_fork+0x1f/0x30 [ 2976.228082] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2976.246955] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2976.250540] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2976.254393] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2976.257147] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2976.264653] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2976.267391] RAX: 0000000000007a89 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2976.269783] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 2976.272351] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 2976.275060] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 2976.277630] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 2976.280350] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 2976.283470] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2976.285294] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 2976.287403] Call Trace: [ 2976.288147] [ 2976.288799] ? show_regs.cold.9+0x22/0x2f [ 2976.289988] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2976.291651] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2976.293236] ? watchdog+0x40/0x40 [ 2976.294355] ? hrtimer_interrupt+0x128/0x2c0 [ 2976.295606] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2976.297330] ? apic_timer_interrupt+0xf/0x20 [ 2976.298919] [ 2976.299549] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2976.301469] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2976.303301] do_raw_spin_lock+0x177/0x190 [ 2976.304581] _raw_spin_lock+0x21/0x30 [ 2976.305771] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2976.307835] ? do_raw_spin_unlock+0x75/0x190 [ 2976.309266] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2976.311565] ? do_wait_intr+0xf0/0xf0 [ 2976.313045] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2976.315386] mdc_reint+0x2d/0x110 [mdc] [ 2976.317080] mdc_unlink+0x658/0x870 [mdc] [ 2976.319398] lmv_unlink+0x34d/0xbd0 [lmv] [ 2976.320988] ll_unlink+0x24d/0x7f0 [lustre] [ 2976.322937] vfs_unlink+0x202/0x330 [ 2976.324333] do_unlinkat+0x2fa/0x460 [ 2976.325442] __x64_sys_unlink+0x37/0x50 [ 2976.327351] do_syscall_64+0xc1/0x440 [ 2976.328778] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2976.331143] RIP: 0033:0x7fe02c11910b [ 2976.332523] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2976.339800] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2976.342507] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 2976.345310] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 2976.347566] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 2976.350357] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 2976.353167] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2980.065010] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 2980.070913] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2980.113380] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2980.120885] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2980.129269] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2980.142134] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2980.147342] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2980.161270] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2980.165875] RAX: 0000000000005967 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 2980.171706] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 2980.177077] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2980.183398] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 2980.188937] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 2980.200159] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 2980.208010] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2980.213335] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 2980.217042] Call Trace: [ 2980.218226] [ 2980.219362] ? show_regs.cold.9+0x22/0x2f [ 2980.225371] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2980.227882] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2980.234396] ? watchdog+0x40/0x40 [ 2980.235723] ? hrtimer_interrupt+0x128/0x2c0 [ 2980.238494] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2980.240768] ? apic_timer_interrupt+0xf/0x20 [ 2980.243027] [ 2980.244084] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2980.246716] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2980.250446] do_raw_spin_lock+0x177/0x190 [ 2980.253659] _raw_spin_lock+0x21/0x30 [ 2980.259278] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2980.266917] process_one_work+0x2c8/0x700 [ 2980.270536] worker_thread+0x296/0x6e0 [ 2980.274325] ? rescuer_thread+0x570/0x570 [ 2980.282374] kthread+0x1d1/0x200 [ 2980.284034] ? set_kthread_struct+0x70/0x70 [ 2980.286874] ret_from_fork+0x1f/0x30 [ 3004.057095] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3004.060024] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3004.063310] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3004.063388] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3004.063396] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3004.063404] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3004.063417] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3004.063422] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3004.063426] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3004.063429] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3004.063430] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3004.063432] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3004.063435] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3004.063438] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3004.063441] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3004.063444] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3004.063452] Call Trace: [ 3004.063458] [ 3004.063464] ? show_regs.cold.9+0x22/0x2f [ 3004.063475] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3004.063482] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3004.063486] ? watchdog+0x40/0x40 [ 3004.063491] ? hrtimer_interrupt+0x128/0x2c0 [ 3004.063495] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3004.063501] ? apic_timer_interrupt+0xf/0x20 [ 3004.063506] [ 3004.063509] ? native_safe_halt+0x2e/0x50 [ 3004.063513] kvm_wait+0xcd/0xe0 [ 3004.063521] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3004.063535] do_raw_spin_lock+0x177/0x190 [ 3004.063541] _raw_spin_lock+0x21/0x30 [ 3004.063546] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3004.071308] Modules linked in: lustre(O) [ 3004.094813] ? do_raw_spin_unlock+0x75/0x190 [ 3004.099808] osp(O) [ 3004.104713] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3004.106757] ofd(O) [ 3004.115206] ? do_wait_intr+0xf0/0xf0 [ 3004.118335] lod(O) [ 3004.121198] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3004.123993] mdt(O) [ 3004.127157] mdc_reint+0x2d/0x110 [mdc] [ 3004.127203] mdc_unlink+0x658/0x870 [mdc] [ 3004.129609] mdd(O) [ 3004.132272] lmv_unlink+0x34d/0xbd0 [lmv] [ 3004.134564] mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3004.134615] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3004.134620] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3004.134622] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3004.134631] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3004.134635] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3004.134638] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3004.134640] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3004.134641] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3004.134642] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3004.134644] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3004.134646] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3004.134648] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3004.134649] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3004.134656] Call Trace: [ 3004.134659] [ 3004.134663] ? show_regs.cold.9+0x22/0x2f [ 3004.134670] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3004.134674] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3004.134676] ? watchdog+0x40/0x40 [ 3004.134679] ? hrtimer_interrupt+0x128/0x2c0 [ 3004.134682] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3004.134686] ? apic_timer_interrupt+0xf/0x20 [ 3004.134690] [ 3004.134691] ? native_safe_halt+0x2e/0x50 [ 3004.134694] kvm_wait+0xcd/0xe0 [ 3004.134699] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3004.134704] do_raw_spin_lock+0x177/0x190 [ 3004.134709] _raw_spin_lock+0x21/0x30 [ 3004.134714] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3004.134986] obd_mod_free+0x46/0x140 [ptlrpc] [ 3004.135250] mdc_commit_open+0x91/0xb0 [mdc] [ 3004.135285] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3004.135483] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3004.135630] after_reply+0xe2c/0x1780 [ptlrpc] [ 3004.135783] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3004.135986] ? __next_timer_interrupt+0x160/0x160 [ 3004.135994] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3004.136258] ? do_wait_intr+0xf0/0xf0 [ 3004.136265] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3004.136514] kthread+0x1d1/0x200 [ 3004.136520] ? set_kthread_struct+0x70/0x70 [ 3004.136523] ret_from_fork+0x1f/0x30 [ 3004.326108] ll_unlink+0x24d/0x7f0 [lustre] [ 3004.327749] vfs_unlink+0x202/0x330 [ 3004.328855] do_unlinkat+0x2fa/0x460 [ 3004.330872] __x64_sys_unlink+0x37/0x50 [ 3004.333515] do_syscall_64+0xc1/0x440 [ 3004.336263] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3004.339310] RIP: 0033:0x7fe02c11910b [ 3004.340867] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3004.350665] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3004.356536] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3004.359716] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3004.363726] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3004.367716] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3004.372732] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3008.065655] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3008.080829] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3008.148229] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3008.159646] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3008.167068] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3008.176459] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3008.185887] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3008.206512] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 3008.218091] RAX: 0000000000004783 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3008.230271] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3008.236522] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3008.249538] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3008.256654] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3008.265048] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3008.277948] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3008.284165] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3008.286508] Call Trace: [ 3008.292094] [ 3008.294484] ? show_regs.cold.9+0x22/0x2f [ 3008.298488] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3008.301655] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3008.311277] ? watchdog+0x40/0x40 [ 3008.313578] ? hrtimer_interrupt+0x128/0x2c0 [ 3008.316960] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3008.319170] ? apic_timer_interrupt+0xf/0x20 [ 3008.324376] [ 3008.326377] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3008.334265] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3008.340460] do_raw_spin_lock+0x177/0x190 [ 3008.342344] _raw_spin_lock+0x21/0x30 [ 3008.344688] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3008.348115] process_one_work+0x2c8/0x700 [ 3008.352167] worker_thread+0x296/0x6e0 [ 3008.357949] ? rescuer_thread+0x570/0x570 [ 3008.360890] kthread+0x1d1/0x200 [ 3008.364262] ? set_kthread_struct+0x70/0x70 [ 3008.368442] ret_from_fork+0x1f/0x30 [ 3032.058205] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3032.060023] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3032.063244] Modules linked in: [ 3032.065614] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3032.065745] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3032.067114] lustre(O) [ 3032.080472] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3032.080491] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3032.080505] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3032.090797] osp(O) [ 3032.091623] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3032.091631] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3032.091633] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3032.091635] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3032.091636] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3032.091638] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3032.091641] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3032.091643] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3032.091645] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3032.091652] Call Trace: [ 3032.091717] [ 3032.091723] ? show_regs.cold.9+0x22/0x2f [ 3032.091737] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3032.091744] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3032.091747] ? watchdog+0x40/0x40 [ 3032.091752] ? hrtimer_interrupt+0x128/0x2c0 [ 3032.091756] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3032.091764] ? apic_timer_interrupt+0xf/0x20 [ 3032.091768] [ 3032.091771] ? native_safe_halt+0x2e/0x50 [ 3032.091775] kvm_wait+0xcd/0xe0 [ 3032.091782] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3032.091789] do_raw_spin_lock+0x177/0x190 [ 3032.091793] _raw_spin_lock+0x21/0x30 [ 3032.091798] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3032.092134] obd_mod_free+0x46/0x140 [ptlrpc] [ 3032.092330] mdc_commit_open+0x91/0xb0 [mdc] [ 3032.092363] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3032.092596] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3032.092925] after_reply+0xe2c/0x1780 [ptlrpc] [ 3032.093130] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3032.093329] ? __next_timer_interrupt+0x160/0x160 [ 3032.093338] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3032.093553] ? do_wait_intr+0xf0/0xf0 [ 3032.093562] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3032.093833] kthread+0x1d1/0x200 [ 3032.093839] ? set_kthread_struct+0x70/0x70 [ 3032.093843] ret_from_fork+0x1f/0x30 [ 3032.312983] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3032.365652] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3032.379957] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3032.388639] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3032.395916] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3032.414607] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 3032.417289] RAX: 0000000000001f0f RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3032.425333] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3032.435331] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3032.438845] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3032.443270] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3032.447127] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3032.450558] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3032.453318] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3032.456445] Call Trace: [ 3032.457750] [ 3032.458661] ? show_regs.cold.9+0x22/0x2f [ 3032.460657] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3032.462831] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3032.465424] ? watchdog+0x40/0x40 [ 3032.466837] ? hrtimer_interrupt+0x128/0x2c0 [ 3032.469137] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3032.471203] ? apic_timer_interrupt+0xf/0x20 [ 3032.473257] [ 3032.474118] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3032.476135] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3032.478184] do_raw_spin_lock+0x177/0x190 [ 3032.481071] _raw_spin_lock+0x21/0x30 [ 3032.482663] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3032.484981] ? do_raw_spin_unlock+0x75/0x190 [ 3032.486830] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3032.489168] ? do_wait_intr+0xf0/0xf0 [ 3032.490784] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3032.493269] mdc_reint+0x2d/0x110 [mdc] [ 3032.494765] mdc_unlink+0x658/0x870 [mdc] [ 3032.495982] lmv_unlink+0x34d/0xbd0 [lmv] [ 3032.497137] ll_unlink+0x24d/0x7f0 [lustre] [ 3032.499029] vfs_unlink+0x202/0x330 [ 3032.500551] do_unlinkat+0x2fa/0x460 [ 3032.502099] __x64_sys_unlink+0x37/0x50 [ 3032.503273] do_syscall_64+0xc1/0x440 [ 3032.504458] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3032.507967] RIP: 0033:0x7fe02c11910b [ 3032.509586] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3032.521228] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3032.525133] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3032.532180] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3032.540106] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3032.550342] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3032.554573] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3036.064371] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3036.073737] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3036.112882] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3036.125897] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3036.130402] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3036.136245] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3036.140551] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3036.157760] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3036.168879] RAX: 0000000000007c90 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3036.177767] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3036.185847] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3036.191283] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3036.194505] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3036.198113] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3036.201424] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3036.204045] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3036.211959] Call Trace: [ 3036.214573] [ 3036.217113] ? show_regs.cold.9+0x22/0x2f [ 3036.221930] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3036.225493] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3036.237264] ? watchdog+0x40/0x40 [ 3036.238420] ? hrtimer_interrupt+0x128/0x2c0 [ 3036.241293] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3036.244234] ? apic_timer_interrupt+0xf/0x20 [ 3036.247941] [ 3036.248475] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3036.249903] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3036.251961] do_raw_spin_lock+0x177/0x190 [ 3036.253401] _raw_spin_lock+0x21/0x30 [ 3036.254903] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3036.256884] process_one_work+0x2c8/0x700 [ 3036.258488] worker_thread+0x296/0x6e0 [ 3036.259898] ? rescuer_thread+0x570/0x570 [ 3036.261640] kthread+0x1d1/0x200 [ 3036.263003] ? set_kthread_struct+0x70/0x70 [ 3036.264728] ret_from_fork+0x1f/0x30 [ 3060.057253] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3060.060018] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3060.061358] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) [ 3060.067454] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3060.067534] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3060.067541] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3060.067544] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3060.067560] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3060.067564] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3060.067568] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3060.067571] RDX: 00000000000014e9 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3060.067573] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3060.067575] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3060.067577] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3060.067580] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3060.067583] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3060.067585] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3060.067593] Call Trace: [ 3060.067599] [ 3060.067605] ? show_regs.cold.9+0x22/0x2f [ 3060.067613] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3060.067619] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3060.067623] ? watchdog+0x40/0x40 [ 3060.067627] ? hrtimer_interrupt+0x128/0x2c0 [ 3060.067631] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3060.067639] ? apic_timer_interrupt+0xf/0x20 [ 3060.067643] [ 3060.067646] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3060.067651] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3060.067655] do_raw_spin_lock+0x177/0x190 [ 3060.067665] _raw_spin_lock+0x21/0x30 [ 3060.067670] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3060.068125] obd_mod_free+0x46/0x140 [ptlrpc] [ 3060.068472] mdc_commit_open+0x91/0xb0 [mdc] [ 3060.079739] rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3060.079785] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3060.079792] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3060.079794] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3060.079809] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3060.079813] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3060.079818] RAX: 000000000000770c RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3060.079820] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3060.079822] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3060.079823] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3060.079825] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3060.079829] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3060.079972] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3060.079975] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3060.079983] Call Trace: [ 3060.079988] [ 3060.080006] ? show_regs.cold.9+0x22/0x2f [ 3060.080016] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3060.080022] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3060.080027] ? watchdog+0x40/0x40 [ 3060.080032] ? hrtimer_interrupt+0x128/0x2c0 [ 3060.080039] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3060.080047] ? apic_timer_interrupt+0xf/0x20 [ 3060.080052] [ 3060.080055] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3060.080060] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3060.080064] do_raw_spin_lock+0x177/0x190 [ 3060.080069] _raw_spin_lock+0x21/0x30 [ 3060.080075] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3060.080546] ? do_raw_spin_unlock+0x75/0x190 [ 3060.080552] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3060.080755] ? do_wait_intr+0xf0/0xf0 [ 3060.080763] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3060.107802] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3060.108628] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3060.116362] mdc_reint+0x2d/0x110 [mdc] [ 3060.116428] mdc_unlink+0x658/0x870 [mdc] [ 3060.116463] lmv_unlink+0x34d/0xbd0 [lmv] [ 3060.116504] ll_unlink+0x24d/0x7f0 [lustre] [ 3060.116637] vfs_unlink+0x202/0x330 [ 3060.116647] do_unlinkat+0x2fa/0x460 [ 3060.116652] __x64_sys_unlink+0x37/0x50 [ 3060.116655] do_syscall_64+0xc1/0x440 [ 3060.116662] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3060.116675] RIP: 0033:0x7fe02c11910b [ 3060.116683] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3060.116687] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3060.116693] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3060.116695] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3060.116697] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3060.116699] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3060.116701] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3060.459622] after_reply+0xe2c/0x1780 [ptlrpc] [ 3060.465266] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3060.469941] ? __next_timer_interrupt+0x160/0x160 [ 3060.476011] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3060.478246] ? do_wait_intr+0xf0/0xf0 [ 3060.481107] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3060.483940] kthread+0x1d1/0x200 [ 3060.488606] ? set_kthread_struct+0x70/0x70 [ 3060.495753] ret_from_fork+0x1f/0x30 [ 3064.064229] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3064.069447] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3064.092678] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3064.097183] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3064.113020] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3064.117579] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3064.121361] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3064.136516] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3064.145457] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3064.157993] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3064.163506] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3064.166740] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3064.172807] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3064.182395] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3064.190196] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3064.194359] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3064.201974] Call Trace: [ 3064.203589] [ 3064.204296] ? show_regs.cold.9+0x22/0x2f [ 3064.205542] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3064.210848] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3064.214642] ? watchdog+0x40/0x40 [ 3064.216229] ? hrtimer_interrupt+0x128/0x2c0 [ 3064.222447] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3064.229226] ? apic_timer_interrupt+0xf/0x20 [ 3064.235794] [ 3064.237281] ? native_safe_halt+0x2e/0x50 [ 3064.239289] kvm_wait+0xcd/0xe0 [ 3064.244225] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3064.249686] do_raw_spin_lock+0x177/0x190 [ 3064.253206] _raw_spin_lock+0x21/0x30 [ 3064.262596] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3064.272233] process_one_work+0x2c8/0x700 [ 3064.273632] worker_thread+0x296/0x6e0 [ 3064.274817] ? rescuer_thread+0x570/0x570 [ 3064.282531] kthread+0x1d1/0x200 [ 3064.286141] ? set_kthread_struct+0x70/0x70 [ 3064.288662] ret_from_fork+0x1f/0x30 [ 3088.057707] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3088.062349] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3088.065293] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3088.065493] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3088.065500] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3088.065513] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3088.065527] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3088.065531] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3088.065535] RAX: 0000000000003786 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3088.065537] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3088.065539] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3088.065541] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3088.065543] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3088.065547] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3088.065549] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3088.065552] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3088.065560] Call Trace: [ 3088.065564] [ 3088.065570] ? show_regs.cold.9+0x22/0x2f [ 3088.065579] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3088.065651] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3088.065656] ? watchdog+0x40/0x40 [ 3088.065660] ? hrtimer_interrupt+0x128/0x2c0 [ 3088.065664] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3088.065672] ? apic_timer_interrupt+0xf/0x20 [ 3088.065685] [ 3088.065990] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3088.065998] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3088.066003] do_raw_spin_lock+0x177/0x190 [ 3088.066008] _raw_spin_lock+0x21/0x30 [ 3088.066014] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3088.066557] ? do_raw_spin_unlock+0x75/0x190 [ 3088.066563] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3088.067447] ? do_wait_intr+0xf0/0xf0 [ 3088.067456] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3088.073175] Modules linked in: lustre(O) osp(O) [ 3088.113444] mdc_reint+0x2d/0x110 [mdc] [ 3088.113496] mdc_unlink+0x658/0x870 [mdc] [ 3088.113544] lmv_unlink+0x34d/0xbd0 [lmv] [ 3088.113577] ll_unlink+0x24d/0x7f0 [lustre] [ 3088.113868] vfs_unlink+0x202/0x330 [ 3088.113880] do_unlinkat+0x2fa/0x460 [ 3088.113886] __x64_sys_unlink+0x37/0x50 [ 3088.113890] do_syscall_64+0xc1/0x440 [ 3088.113897] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3088.113905] RIP: 0033:0x7fe02c11910b [ 3088.114111] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3088.114123] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3088.114129] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3088.114132] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3088.114134] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3088.114136] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3088.114138] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3088.433634] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3088.493028] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3088.508128] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3088.520241] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3088.536324] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3088.559119] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3088.574232] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3088.581251] RDX: 0000000000003e9f RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3088.585765] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3088.592761] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3088.603102] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3088.607948] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3088.619761] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3088.630868] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3088.644033] Call Trace: [ 3088.646798] [ 3088.652694] ? show_regs.cold.9+0x22/0x2f [ 3088.656478] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3088.662245] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3088.666470] ? watchdog+0x40/0x40 [ 3088.671369] ? hrtimer_interrupt+0x128/0x2c0 [ 3088.682287] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3088.691967] ? apic_timer_interrupt+0xf/0x20 [ 3088.707950] [ 3088.708759] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3088.725047] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3088.730694] do_raw_spin_lock+0x177/0x190 [ 3088.733592] _raw_spin_lock+0x21/0x30 [ 3088.735858] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3088.739041] obd_mod_free+0x46/0x140 [ptlrpc] [ 3088.741570] mdc_commit_open+0x91/0xb0 [mdc] [ 3088.743446] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3088.746869] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3088.749647] after_reply+0xe2c/0x1780 [ptlrpc] [ 3088.751787] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3088.759732] ? __next_timer_interrupt+0x160/0x160 [ 3088.765449] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3088.769357] ? do_wait_intr+0xf0/0xf0 [ 3088.771669] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3088.774801] kthread+0x1d1/0x200 [ 3088.776652] ? set_kthread_struct+0x70/0x70 [ 3088.779052] ret_from_fork+0x1f/0x30 [ 3092.071092] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3092.085324] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3092.114580] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3092.124809] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3092.130523] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3092.135080] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3092.139054] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3092.151680] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3092.156955] RAX: 0000000000005884 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3092.161550] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3092.165778] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3092.170405] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3092.174771] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3092.179780] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3092.185760] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3092.189504] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3092.194665] Call Trace: [ 3092.196515] [ 3092.197951] ? show_regs.cold.9+0x22/0x2f [ 3092.200866] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3092.203966] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3092.207051] ? watchdog+0x40/0x40 [ 3092.209449] ? hrtimer_interrupt+0x128/0x2c0 [ 3092.212173] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3092.215653] ? apic_timer_interrupt+0xf/0x20 [ 3092.219715] [ 3092.221409] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3092.224799] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3092.228489] do_raw_spin_lock+0x177/0x190 [ 3092.231144] _raw_spin_lock+0x21/0x30 [ 3092.233670] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3092.238044] process_one_work+0x2c8/0x700 [ 3092.242142] worker_thread+0x296/0x6e0 [ 3092.245083] ? rescuer_thread+0x570/0x570 [ 3092.247283] kthread+0x1d1/0x200 [ 3092.249207] ? set_kthread_struct+0x70/0x70 [ 3092.252220] ret_from_fork+0x1f/0x30 [ 3102.120765] rcu: INFO: rcu_sched self-detected stall on CPU [ 3102.127311] rcu: 1-....: (625560 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=181826 [ 3102.135987] (t=960028 jiffies g=278549 q=6082) [ 3102.140490] Sending NMI from CPU 1 to CPUs 0: [ 3102.159832] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 8.316 msecs [ 3102.165791] NMI backtrace for cpu 0 [ 3102.165807] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3102.165809] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3102.165811] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3102.165812] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3102.165815] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 [ 3102.165818] RAX: 0000000000006713 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3102.165819] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3102.165820] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3102.165821] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3102.165823] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3102.165824] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3102.165825] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3102.165827] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3102.165828] Call Trace: [ 3102.165829] [ 3102.165830] ? show_regs.cold.9+0x22/0x2f [ 3102.165831] ? nmi_cpu_backtrace.cold.8+0x98/0xa2 [ 3102.165832] ? nmi_cpu_backtrace_handler+0x15/0x40 [ 3102.165833] ? nmi_handle+0xbd/0x210 [ 3102.165834] ? default_do_nmi+0x4b/0x1b0 [ 3102.165835] ? do_nmi+0x280/0x380 [ 3102.165836] ? end_repeat_nmi+0x16/0x1f [ 3102.165837] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3102.165838] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3102.165839] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3102.165840] [ 3102.165840] do_raw_spin_lock+0x177/0x190 [ 3102.165841] _raw_spin_lock+0x21/0x30 [ 3102.165842] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3102.165843] ? do_raw_spin_unlock+0x75/0x190 [ 3102.165844] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3102.165845] ? do_wait_intr+0xf0/0xf0 [ 3102.165845] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3102.165846] mdc_reint+0x2d/0x110 [mdc] [ 3102.165847] mdc_unlink+0x658/0x870 [mdc] [ 3102.165848] lmv_unlink+0x34d/0xbd0 [lmv] [ 3102.165849] ll_unlink+0x24d/0x7f0 [lustre] [ 3102.165850] vfs_unlink+0x202/0x330 [ 3102.165851] do_unlinkat+0x2fa/0x460 [ 3102.165851] __x64_sys_unlink+0x37/0x50 [ 3102.165852] do_syscall_64+0xc1/0x440 [ 3102.165853] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3102.165854] RIP: 0033:0x7fe02c11910b [ 3102.165855] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3102.165857] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3102.165859] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3102.165860] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3102.165861] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3102.165862] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3102.165863] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3102.165867] NMI backtrace for cpu 1 [ 3102.613995] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3102.625367] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3102.635097] Call Trace: [ 3102.639736] [ 3102.640464] ? dump_stack+0xbb/0x10e [ 3102.641702] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3102.651525] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3102.656040] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3102.674302] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3102.687981] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3102.689908] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3102.700849] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3102.722177] ? update_process_times+0x6d/0xc0 [ 3102.723797] ? tick_sched_handle.isra.18+0x27/0x90 [ 3102.741045] ? tick_sched_timer+0x53/0xb0 [ 3102.756699] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3102.758417] ? tick_sched_handle.isra.18+0x90/0x90 [ 3102.759973] ? hrtimer_interrupt+0x128/0x2c0 [ 3102.781046] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3102.796735] ? apic_timer_interrupt+0xf/0x20 [ 3102.806597] [ 3102.817064] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3102.819068] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3102.846333] ? do_raw_spin_lock+0x177/0x190 [ 3102.854929] ? _raw_spin_lock+0x21/0x30 [ 3102.860855] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3102.863339] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3102.880945] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3102.889674] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3102.898442] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3102.904422] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3102.919045] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3102.920993] ? __next_timer_interrupt+0x160/0x160 [ 3102.935883] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3102.959030] ? do_wait_intr+0xf0/0xf0 [ 3102.960065] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3102.978195] ? kthread+0x1d1/0x200 [ 3102.980454] ? set_kthread_struct+0x70/0x70 [ 3102.986508] ? ret_from_fork+0x1f/0x30 [ 3120.064434] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3120.069528] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3120.112082] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3120.121856] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3120.129363] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3120.132902] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3120.136559] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3120.149178] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3120.156598] RAX: 000000000000288d RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3120.164927] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3120.171408] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3120.181004] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3120.190237] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3120.195460] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3120.200452] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3120.202933] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3120.218927] Call Trace: [ 3120.221401] [ 3120.222852] ? show_regs.cold.9+0x22/0x2f [ 3120.225315] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3120.229311] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3120.236147] ? watchdog+0x40/0x40 [ 3120.237917] ? hrtimer_interrupt+0x128/0x2c0 [ 3120.242302] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3120.247275] ? apic_timer_interrupt+0xf/0x20 [ 3120.249980] [ 3120.251530] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3120.256539] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3120.262030] do_raw_spin_lock+0x177/0x190 [ 3120.265339] _raw_spin_lock+0x21/0x30 [ 3120.267537] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3120.274326] process_one_work+0x2c8/0x700 [ 3120.275725] worker_thread+0x296/0x6e0 [ 3120.282868] ? rescuer_thread+0x570/0x570 [ 3120.286432] kthread+0x1d1/0x200 [ 3120.288561] ? set_kthread_struct+0x70/0x70 [ 3120.291244] ret_from_fork+0x1f/0x30 [ 3128.058988] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3128.065051] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3128.067609] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3128.108782] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3128.108797] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3128.108810] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3128.108825] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3128.108830] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3128.108837] RAX: 0000000000007c75 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3128.116000] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3128.116076] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3128.116084] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3128.116087] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3128.116100] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3128.116103] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3128.116107] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3128.116109] RDX: 0000000000000a0b RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3128.116112] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3128.116113] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3128.116115] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3128.116118] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3128.116120] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3128.116122] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3128.116131] Call Trace: [ 3128.116136] [ 3128.116142] ? show_regs.cold.9+0x22/0x2f [ 3128.116150] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3128.116156] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3128.116160] ? watchdog+0x40/0x40 [ 3128.116164] ? hrtimer_interrupt+0x128/0x2c0 [ 3128.116168] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3128.116176] ? apic_timer_interrupt+0xf/0x20 [ 3128.116180] [ 3128.116183] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3128.116187] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3128.117415] do_raw_spin_lock+0x177/0x190 [ 3128.117429] _raw_spin_lock+0x21/0x30 [ 3128.117437] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3128.118668] obd_mod_free+0x46/0x140 [ptlrpc] [ 3128.120379] mdc_commit_open+0x91/0xb0 [mdc] [ 3128.120418] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3128.120953] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3128.121489] after_reply+0xe2c/0x1780 [ptlrpc] [ 3128.136006] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3128.136016] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3128.136019] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3128.136023] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3128.136027] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3128.136031] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3128.136033] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3128.136041] Call Trace: [ 3128.136048] [ 3128.136053] ? show_regs.cold.9+0x22/0x2f [ 3128.136065] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3128.136072] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3128.136076] ? watchdog+0x40/0x40 [ 3128.136080] ? hrtimer_interrupt+0x128/0x2c0 [ 3128.136084] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3128.136093] ? apic_timer_interrupt+0xf/0x20 [ 3128.136097] [ 3128.136100] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3128.136107] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3128.136111] do_raw_spin_lock+0x177/0x190 [ 3128.136123] _raw_spin_lock+0x21/0x30 [ 3128.136130] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3128.136620] ? do_raw_spin_unlock+0x75/0x190 [ 3128.136628] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3128.136854] ? do_wait_intr+0xf0/0xf0 [ 3128.136861] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3128.137452] mdc_reint+0x2d/0x110 [mdc] [ 3128.137492] mdc_unlink+0x658/0x870 [mdc] [ 3128.137517] lmv_unlink+0x34d/0xbd0 [lmv] [ 3128.137548] ll_unlink+0x24d/0x7f0 [lustre] [ 3128.137663] vfs_unlink+0x202/0x330 [ 3128.137672] do_unlinkat+0x2fa/0x460 [ 3128.137677] __x64_sys_unlink+0x37/0x50 [ 3128.137681] do_syscall_64+0xc1/0x440 [ 3128.137688] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3128.137695] RIP: 0033:0x7fe02c11910b [ 3128.138294] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3128.138301] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3128.138308] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3128.138310] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3128.138312] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3128.138314] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3128.138316] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3128.543485] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3128.550313] ? __next_timer_interrupt+0x160/0x160 [ 3128.557332] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3128.561442] ? do_wait_intr+0xf0/0xf0 [ 3128.566809] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3128.569343] kthread+0x1d1/0x200 [ 3128.570715] ? set_kthread_struct+0x70/0x70 [ 3128.571847] ret_from_fork+0x1f/0x30 [ 3148.070819] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3148.082568] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3148.144667] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3148.150238] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3148.154486] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3148.158057] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3148.165637] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3148.181562] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 3148.185922] RAX: 000000000000738b RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3148.191462] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3148.196562] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3148.199953] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3148.203830] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3148.212565] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3148.218327] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3148.222152] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3148.229332] Call Trace: [ 3148.230589] [ 3148.231941] ? show_regs.cold.9+0x22/0x2f [ 3148.233638] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3148.236766] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3148.240570] ? watchdog+0x40/0x40 [ 3148.242237] ? hrtimer_interrupt+0x128/0x2c0 [ 3148.244815] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3148.246951] ? apic_timer_interrupt+0xf/0x20 [ 3148.248936] [ 3148.249646] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3148.253286] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3148.255769] do_raw_spin_lock+0x177/0x190 [ 3148.257550] _raw_spin_lock+0x21/0x30 [ 3148.260194] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3148.262334] process_one_work+0x2c8/0x700 [ 3148.265337] worker_thread+0x296/0x6e0 [ 3148.267051] ? rescuer_thread+0x570/0x570 [ 3148.270374] kthread+0x1d1/0x200 [ 3148.271701] ? set_kthread_struct+0x70/0x70 [ 3148.274727] ret_from_fork+0x1f/0x30 [ 3156.057223] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3156.062981] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3156.063939] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3156.095198] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3156.095211] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3156.095214] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3156.095227] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3156.095231] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3156.095237] RAX: 0000000000004414 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3156.095239] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3156.095241] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3156.095243] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3156.095245] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3156.095248] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3156.095250] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3156.095252] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3156.095260] Call Trace: [ 3156.095266] [ 3156.095271] ? show_regs.cold.9+0x22/0x2f [ 3156.095293] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3156.095300] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3156.095304] ? watchdog+0x40/0x40 [ 3156.095308] ? hrtimer_interrupt+0x128/0x2c0 [ 3156.095321] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3156.095329] ? apic_timer_interrupt+0xf/0x20 [ 3156.095334] [ 3156.095337] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3156.095342] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3156.095347] do_raw_spin_lock+0x177/0x190 [ 3156.095352] _raw_spin_lock+0x21/0x30 [ 3156.095360] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3156.096859] ? do_raw_spin_unlock+0x75/0x190 [ 3156.096866] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3156.097017] ? do_wait_intr+0xf0/0xf0 [ 3156.097022] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3156.097157] mdc_reint+0x2d/0x110 [mdc] [ 3156.097192] mdc_unlink+0x658/0x870 [mdc] [ 3156.097209] lmv_unlink+0x34d/0xbd0 [lmv] [ 3156.097229] ll_unlink+0x24d/0x7f0 [lustre] [ 3156.097335] vfs_unlink+0x202/0x330 [ 3156.097342] do_unlinkat+0x2fa/0x460 [ 3156.097347] __x64_sys_unlink+0x37/0x50 [ 3156.097351] do_syscall_64+0xc1/0x440 [ 3156.097357] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3156.097363] RIP: 0033:0x7fe02c11910b [ 3156.097370] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3156.097373] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3156.097378] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3156.097380] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3156.097382] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3156.097384] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3156.097386] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3156.386815] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3156.440557] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3156.456252] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3156.465408] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3156.471224] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3156.485296] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3156.499128] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3156.505634] RDX: 00000000000048d3 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3156.511205] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3156.525835] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3156.533594] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3156.540588] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3156.555917] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3156.565016] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3156.569767] Call Trace: [ 3156.572616] [ 3156.573327] ? show_regs.cold.9+0x22/0x2f [ 3156.574802] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3156.584993] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3156.593961] ? watchdog+0x40/0x40 [ 3156.600356] ? hrtimer_interrupt+0x128/0x2c0 [ 3156.613384] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3156.617206] ? apic_timer_interrupt+0xf/0x20 [ 3156.627591] [ 3156.633225] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3156.644582] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3156.649831] do_raw_spin_lock+0x177/0x190 [ 3156.655076] _raw_spin_lock+0x21/0x30 [ 3156.663450] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3156.668177] obd_mod_free+0x46/0x140 [ptlrpc] [ 3156.671798] mdc_commit_open+0x91/0xb0 [mdc] [ 3156.672976] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3156.679080] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3156.687125] after_reply+0xe2c/0x1780 [ptlrpc] [ 3156.691632] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3156.699106] ? __next_timer_interrupt+0x160/0x160 [ 3156.706656] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3156.715524] ? do_wait_intr+0xf0/0xf0 [ 3156.729500] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3156.734818] kthread+0x1d1/0x200 [ 3156.742055] ? set_kthread_struct+0x70/0x70 [ 3156.747446] ret_from_fork+0x1f/0x30 [ 3176.064017] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3176.068171] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3176.115218] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3176.129516] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3176.144075] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3176.149056] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3176.152038] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3176.163160] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3176.168731] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3176.171032] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3176.175576] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3176.178418] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3176.181976] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3176.186245] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3176.189655] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3176.192075] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3176.195146] Call Trace: [ 3176.196534] [ 3176.197436] ? show_regs.cold.9+0x22/0x2f [ 3176.199055] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3176.203279] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3176.206338] ? watchdog+0x40/0x40 [ 3176.207527] ? hrtimer_interrupt+0x128/0x2c0 [ 3176.208680] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3176.212512] ? apic_timer_interrupt+0xf/0x20 [ 3176.214676] [ 3176.217382] ? native_safe_halt+0x2e/0x50 [ 3176.218443] kvm_wait+0xcd/0xe0 [ 3176.219266] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3176.220931] do_raw_spin_lock+0x177/0x190 [ 3176.223052] _raw_spin_lock+0x21/0x30 [ 3176.225496] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3176.227912] process_one_work+0x2c8/0x700 [ 3176.230543] worker_thread+0x296/0x6e0 [ 3176.233236] ? rescuer_thread+0x570/0x570 [ 3176.235334] kthread+0x1d1/0x200 [ 3176.237156] ? set_kthread_struct+0x70/0x70 [ 3176.239623] ret_from_fork+0x1f/0x30 [ 3184.057027] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3184.059790] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3184.060069] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3184.103494] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3184.106463] Modules linked in: lustre(O) [ 3184.110513] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3184.110520] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3184.110608] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3184.110617] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3184.112144] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3184.116282] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3184.141362] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3184.146188] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3184.146202] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3184.146208] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3184.146210] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3184.146212] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3184.146214] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3184.146216] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3184.146218] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3184.146220] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3184.146222] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3184.146230] Call Trace: [ 3184.146241] [ 3184.146247] ? show_regs.cold.9+0x22/0x2f [ 3184.146258] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3184.146265] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3184.146269] ? watchdog+0x40/0x40 [ 3184.146273] ? hrtimer_interrupt+0x128/0x2c0 [ 3184.146277] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3184.146285] ? apic_timer_interrupt+0xf/0x20 [ 3184.146290] [ 3184.146293] ? native_safe_halt+0x2e/0x50 [ 3184.146298] kvm_wait+0xcd/0xe0 [ 3184.146311] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3184.146319] do_raw_spin_lock+0x177/0x190 [ 3184.146323] _raw_spin_lock+0x21/0x30 [ 3184.146333] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3184.153058] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3184.153070] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3184.153072] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3184.153074] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3184.153076] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3184.153078] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3184.153080] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3184.153083] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3184.153084] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3184.153091] Call Trace: [ 3184.153102] [ 3184.153107] ? show_regs.cold.9+0x22/0x2f [ 3184.153120] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3184.155173] obd_mod_free+0x46/0x140 [ptlrpc] [ 3184.155482] mdc_commit_open+0x91/0xb0 [mdc] [ 3184.157681] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3184.166486] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3184.182767] ? watchdog+0x40/0x40 [ 3184.185555] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3184.190061] ? hrtimer_interrupt+0x128/0x2c0 [ 3184.190078] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3184.190085] ? apic_timer_interrupt+0xf/0x20 [ 3184.190089] [ 3184.190093] ? native_safe_halt+0x2e/0x50 [ 3184.190097] kvm_wait+0xcd/0xe0 [ 3184.190107] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3184.190114] do_raw_spin_lock+0x177/0x190 [ 3184.190118] _raw_spin_lock+0x21/0x30 [ 3184.190123] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3184.193844] after_reply+0xe2c/0x1780 [ptlrpc] [ 3184.197691] ? do_raw_spin_unlock+0x75/0x190 [ 3184.200744] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3184.210770] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3184.214969] ? __next_timer_interrupt+0x160/0x160 [ 3184.219141] ? do_wait_intr+0xf0/0xf0 [ 3184.222641] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3184.223093] ? do_wait_intr+0xf0/0xf0 [ 3184.223104] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3184.223393] kthread+0x1d1/0x200 [ 3184.223400] ? set_kthread_struct+0x70/0x70 [ 3184.223405] ret_from_fork+0x1f/0x30 [ 3184.414054] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3184.417544] mdc_reint+0x2d/0x110 [mdc] [ 3184.420151] mdc_unlink+0x658/0x870 [mdc] [ 3184.422508] lmv_unlink+0x34d/0xbd0 [lmv] [ 3184.425384] ll_unlink+0x24d/0x7f0 [lustre] [ 3184.428805] vfs_unlink+0x202/0x330 [ 3184.431197] do_unlinkat+0x2fa/0x460 [ 3184.433758] __x64_sys_unlink+0x37/0x50 [ 3184.439532] do_syscall_64+0xc1/0x440 [ 3184.441763] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3184.445351] RIP: 0033:0x7fe02c11910b [ 3184.447880] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3184.458118] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3184.463078] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3184.467492] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3184.470806] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3184.475701] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3184.479936] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3204.064027] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3204.073589] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3204.131458] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3204.142399] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3204.150314] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3204.157259] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3204.163240] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3204.173972] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3204.179268] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3204.184798] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3204.189105] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3204.193681] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3204.197929] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3204.202497] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3204.210617] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3204.213776] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3204.217912] Call Trace: [ 3204.219065] [ 3204.220135] ? show_regs.cold.9+0x22/0x2f [ 3204.222996] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3204.228312] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3204.231174] ? watchdog+0x40/0x40 [ 3204.233967] ? hrtimer_interrupt+0x128/0x2c0 [ 3204.236143] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3204.240718] ? apic_timer_interrupt+0xf/0x20 [ 3204.244359] [ 3204.246098] ? native_safe_halt+0x2e/0x50 [ 3204.248192] kvm_wait+0xcd/0xe0 [ 3204.250087] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3204.253198] do_raw_spin_lock+0x177/0x190 [ 3204.255673] _raw_spin_lock+0x21/0x30 [ 3204.257195] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3204.263169] process_one_work+0x2c8/0x700 [ 3204.267410] worker_thread+0x296/0x6e0 [ 3204.271100] ? rescuer_thread+0x570/0x570 [ 3204.279850] kthread+0x1d1/0x200 [ 3204.287676] ? set_kthread_struct+0x70/0x70 [ 3204.295006] ret_from_fork+0x1f/0x30 [ 3212.059255] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3212.060029] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3212.063151] Modules linked in: [ 3212.066950] Modules linked in: [ 3212.066953] lustre(O) osp(O) [ 3212.068275] lustre(O) [ 3212.069701] ofd(O) [ 3212.071215] osp(O) [ 3212.072147] lod(O) [ 3212.073205] ofd(O) [ 3212.074038] mdt(O) mdd(O) [ 3212.074839] lod(O) [ 3212.075423] mgs(O) [ 3212.076254] mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3212.076339] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3212.076345] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3212.077102] lquota(O) [ 3212.078161] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3212.093780] lfsck(O) [ 3212.098587] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3212.101282] obdecho(O) [ 3212.102050] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 [ 3212.103410] mgc(O) [ 3212.104213] ORIG_RAX: ffffffffffffff13 [ 3212.109872] mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 [ 3212.111151] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3212.112630] sha512_generic [ 3212.113565] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3212.113570] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3212.113573] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3212.113576] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3212.113579] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3212.113582] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3212.113584] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3212.113593] Call Trace: [ 3212.113600] [ 3212.115050] [ 3212.115058] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3212.131351] ? show_regs.cold.9+0x22/0x2f [ 3212.134142] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3212.135457] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3212.138177] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3212.141577] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3212.143925] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3212.147352] ? watchdog+0x40/0x40 [ 3212.150478] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 [ 3212.152834] ? hrtimer_interrupt+0x128/0x2c0 [ 3212.155427] ORIG_RAX: ffffffffffffff13 [ 3212.156656] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3212.157320] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3212.157973] ? apic_timer_interrupt+0xf/0x20 [ 3212.162278] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3212.164391] [ 3212.167919] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3212.170373] ? native_safe_halt+0x2e/0x50 [ 3212.172269] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3212.174724] kvm_wait+0xcd/0xe0 [ 3212.174744] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3212.174752] do_raw_spin_lock+0x177/0x190 [ 3212.174757] _raw_spin_lock+0x21/0x30 [ 3212.174765] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3212.181199] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3212.182618] obd_mod_free+0x46/0x140 [ptlrpc] [ 3212.184199] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3212.184205] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3212.184207] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3212.184214] Call Trace: [ 3212.184220] [ 3212.184224] ? show_regs.cold.9+0x22/0x2f [ 3212.184235] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3212.184242] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3212.184246] ? watchdog+0x40/0x40 [ 3212.184250] ? hrtimer_interrupt+0x128/0x2c0 [ 3212.184254] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3212.184261] ? apic_timer_interrupt+0xf/0x20 [ 3212.184266] [ 3212.184268] ? native_safe_halt+0x2e/0x50 [ 3212.184272] kvm_wait+0xcd/0xe0 [ 3212.184279] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3212.184285] do_raw_spin_lock+0x177/0x190 [ 3212.184289] _raw_spin_lock+0x21/0x30 [ 3212.184294] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3212.186990] mdc_commit_open+0x91/0xb0 [mdc] [ 3212.188196] ? do_raw_spin_unlock+0x75/0x190 [ 3212.191111] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3212.193364] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3212.195657] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3212.197823] ? do_wait_intr+0xf0/0xf0 [ 3212.197837] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3212.198078] mdc_reint+0x2d/0x110 [mdc] [ 3212.198114] mdc_unlink+0x658/0x870 [mdc] [ 3212.198140] lmv_unlink+0x34d/0xbd0 [lmv] [ 3212.198167] ll_unlink+0x24d/0x7f0 [lustre] [ 3212.198272] vfs_unlink+0x202/0x330 [ 3212.198279] do_unlinkat+0x2fa/0x460 [ 3212.198284] __x64_sys_unlink+0x37/0x50 [ 3212.198286] do_syscall_64+0xc1/0x440 [ 3212.198292] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3212.198300] RIP: 0033:0x7fe02c11910b [ 3212.198307] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3212.198311] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3212.198317] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3212.199031] after_reply+0xe2c/0x1780 [ptlrpc] [ 3212.201171] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3212.203065] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3212.205372] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3212.206424] ? __next_timer_interrupt+0x160/0x160 [ 3212.208559] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3212.209702] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3212.211243] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3212.212414] ? do_wait_intr+0xf0/0xf0 [ 3212.295574] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3212.297523] kthread+0x1d1/0x200 [ 3212.298557] ? set_kthread_struct+0x70/0x70 [ 3212.300459] ret_from_fork+0x1f/0x30 [ 3232.066594] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3232.070535] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3232.113634] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3232.122140] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3232.132056] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3232.136584] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3232.139807] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3232.150627] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3232.156045] RAX: 0000000000007e8e RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3232.160770] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3232.164555] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3232.170134] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3232.176728] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3232.182285] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3232.187423] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3232.190347] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3232.195768] Call Trace: [ 3232.196940] [ 3232.198070] ? show_regs.cold.9+0x22/0x2f [ 3232.201488] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3232.206444] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3232.208997] ? watchdog+0x40/0x40 [ 3232.211067] ? hrtimer_interrupt+0x128/0x2c0 [ 3232.214040] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3232.216882] ? apic_timer_interrupt+0xf/0x20 [ 3232.221621] [ 3232.225098] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3232.228607] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3232.231184] do_raw_spin_lock+0x177/0x190 [ 3232.235554] _raw_spin_lock+0x21/0x30 [ 3232.237130] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3232.240476] process_one_work+0x2c8/0x700 [ 3232.241883] worker_thread+0x296/0x6e0 [ 3232.245072] ? rescuer_thread+0x570/0x570 [ 3232.247056] kthread+0x1d1/0x200 [ 3232.248790] ? set_kthread_struct+0x70/0x70 [ 3232.252830] ret_from_fork+0x1f/0x30 [ 3240.059006] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3240.060063] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3240.070661] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3240.070733] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3240.070740] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3240.070743] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3240.070757] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3240.070761] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3240.070765] RAX: 0000000000007c84 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3240.070768] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3240.070770] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3240.070772] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3240.070774] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3240.070777] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3240.070780] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3240.070782] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3240.070790] Call Trace: [ 3240.070795] [ 3240.070800] ? show_regs.cold.9+0x22/0x2f [ 3240.070818] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3240.070824] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3240.070828] ? watchdog+0x40/0x40 [ 3240.070833] ? hrtimer_interrupt+0x128/0x2c0 [ 3240.070837] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3240.076974] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) [ 3240.140744] ? apic_timer_interrupt+0xf/0x20 [ 3240.140765] [ 3240.140769] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3240.140777] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3240.140782] do_raw_spin_lock+0x177/0x190 [ 3240.140788] _raw_spin_lock+0x21/0x30 [ 3240.140796] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3240.154450] lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3240.154512] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3240.154520] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3240.154523] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3240.154538] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3240.154542] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3240.154547] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3240.154549] RDX: 0000000000005791 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3240.154551] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3240.154553] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3240.154556] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3240.154559] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3240.154562] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3240.154564] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3240.154574] Call Trace: [ 3240.154580] [ 3240.154586] ? show_regs.cold.9+0x22/0x2f [ 3240.154596] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3240.154601] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3240.154606] ? watchdog+0x40/0x40 [ 3240.154610] ? hrtimer_interrupt+0x128/0x2c0 [ 3240.154614] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3240.154623] ? apic_timer_interrupt+0xf/0x20 [ 3240.154628] [ 3240.154631] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3240.154635] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3240.154640] do_raw_spin_lock+0x177/0x190 [ 3240.154645] _raw_spin_lock+0x21/0x30 [ 3240.154651] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3240.154936] obd_mod_free+0x46/0x140 [ptlrpc] [ 3240.166801] ? do_raw_spin_unlock+0x75/0x190 [ 3240.166823] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3240.167563] ? do_wait_intr+0xf0/0xf0 [ 3240.167574] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3240.167784] mdc_reint+0x2d/0x110 [mdc] [ 3240.167825] mdc_unlink+0x658/0x870 [mdc] [ 3240.167863] lmv_unlink+0x34d/0xbd0 [lmv] [ 3240.167893] ll_unlink+0x24d/0x7f0 [lustre] [ 3240.168026] vfs_unlink+0x202/0x330 [ 3240.168036] do_unlinkat+0x2fa/0x460 [ 3240.168041] __x64_sys_unlink+0x37/0x50 [ 3240.168046] do_syscall_64+0xc1/0x440 [ 3240.168054] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3240.168063] RIP: 0033:0x7fe02c11910b [ 3240.168071] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3240.168075] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3240.168081] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3240.168084] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3240.168085] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3240.168087] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3240.168089] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3240.661533] mdc_commit_open+0x91/0xb0 [mdc] [ 3240.666919] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3240.669649] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3240.677416] after_reply+0xe2c/0x1780 [ptlrpc] [ 3240.683811] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3240.688873] ? __next_timer_interrupt+0x160/0x160 [ 3240.692343] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3240.694074] ? do_wait_intr+0xf0/0xf0 [ 3240.699863] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3240.709795] kthread+0x1d1/0x200 [ 3240.713870] ? set_kthread_struct+0x70/0x70 [ 3240.719369] ret_from_fork+0x1f/0x30 [ 3260.064524] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3260.071350] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3260.111182] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3260.116868] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3260.121184] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3260.127869] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3260.135257] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3260.150278] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 3260.155229] RAX: 00000000000077a8 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3260.159285] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3260.163770] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3260.173802] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3260.181334] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3260.189506] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3260.197271] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3260.210527] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3260.218809] Call Trace: [ 3260.222184] [ 3260.222903] ? show_regs.cold.9+0x22/0x2f [ 3260.224329] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3260.233220] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3260.249329] ? watchdog+0x40/0x40 [ 3260.250480] ? hrtimer_interrupt+0x128/0x2c0 [ 3260.252016] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3260.274870] ? apic_timer_interrupt+0xf/0x20 [ 3260.287304] [ 3260.287987] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3260.301370] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3260.303313] do_raw_spin_lock+0x177/0x190 [ 3260.323984] _raw_spin_lock+0x21/0x30 [ 3260.325324] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3260.335706] process_one_work+0x2c8/0x700 [ 3260.337069] worker_thread+0x296/0x6e0 [ 3260.338135] ? rescuer_thread+0x570/0x570 [ 3260.352706] kthread+0x1d1/0x200 [ 3260.358636] ? set_kthread_struct+0x70/0x70 [ 3260.361537] ret_from_fork+0x1f/0x30 [ 3268.057231] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3268.060537] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3268.062011] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3268.082819] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3268.082838] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3268.082842] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3268.082857] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3268.082866] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3268.082876] RAX: 000000000000337a RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3268.082879] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3268.082885] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3268.082887] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3268.082889] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3268.082896] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3268.082899] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3268.082905] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3268.082920] Call Trace: [ 3268.082929] [ 3268.082934] ? show_regs.cold.9+0x22/0x2f [ 3268.082950] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3268.082963] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3268.082968] ? watchdog+0x40/0x40 [ 3268.082976] ? hrtimer_interrupt+0x128/0x2c0 [ 3268.082984] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3268.082997] ? apic_timer_interrupt+0xf/0x20 [ 3268.083005] [ 3268.083008] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3268.083017] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3268.083025] do_raw_spin_lock+0x177/0x190 [ 3268.083030] _raw_spin_lock+0x21/0x30 [ 3268.083044] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3268.083929] ? do_raw_spin_unlock+0x75/0x190 [ 3268.083937] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3268.084170] ? do_wait_intr+0xf0/0xf0 [ 3268.084179] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3268.084425] mdc_reint+0x2d/0x110 [mdc] [ 3268.084478] mdc_unlink+0x658/0x870 [mdc] [ 3268.084507] lmv_unlink+0x34d/0xbd0 [lmv] [ 3268.084552] ll_unlink+0x24d/0x7f0 [lustre] [ 3268.084717] vfs_unlink+0x202/0x330 [ 3268.084725] do_unlinkat+0x2fa/0x460 [ 3268.084729] __x64_sys_unlink+0x37/0x50 [ 3268.084733] do_syscall_64+0xc1/0x440 [ 3268.084739] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3268.084747] RIP: 0033:0x7fe02c11910b [ 3268.084754] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3268.084758] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3268.084763] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3268.084765] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3268.084767] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3268.084769] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3268.084771] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3268.414614] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3268.462608] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3268.479627] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3268.486581] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3268.488304] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3268.499206] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3268.504048] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3268.510136] RDX: 00000000000009a0 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3268.525130] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3268.530247] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3268.535273] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3268.539684] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3268.548111] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3268.551177] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3268.560659] Call Trace: [ 3268.563510] [ 3268.568874] ? show_regs.cold.9+0x22/0x2f [ 3268.572643] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3268.576187] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3268.577641] ? watchdog+0x40/0x40 [ 3268.589294] ? hrtimer_interrupt+0x128/0x2c0 [ 3268.594614] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3268.598175] ? apic_timer_interrupt+0xf/0x20 [ 3268.604089] [ 3268.605496] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3268.610971] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3268.617200] do_raw_spin_lock+0x177/0x190 [ 3268.621153] _raw_spin_lock+0x21/0x30 [ 3268.623016] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3268.626084] obd_mod_free+0x46/0x140 [ptlrpc] [ 3268.628258] mdc_commit_open+0x91/0xb0 [mdc] [ 3268.642371] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3268.650075] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3268.656372] after_reply+0xe2c/0x1780 [ptlrpc] [ 3268.663152] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3268.672680] ? __next_timer_interrupt+0x160/0x160 [ 3268.677799] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3268.684121] ? do_wait_intr+0xf0/0xf0 [ 3268.688338] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3268.693103] kthread+0x1d1/0x200 [ 3268.697014] ? set_kthread_struct+0x70/0x70 [ 3268.710114] ret_from_fork+0x1f/0x30 [ 3282.124736] rcu: INFO: rcu_sched self-detected stall on CPU [ 3282.128975] rcu: 1-....: (735274 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=215128 [ 3282.138807] (t=1140032 jiffies g=278549 q=7370) [ 3282.142811] Sending NMI from CPU 1 to CPUs 0: [ 3282.147566] NMI backtrace for cpu 0 [ 3282.147573] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3282.147575] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3282.147576] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3282.147578] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3282.147580] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 [ 3282.147583] RAX: 0000000000001394 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3282.147584] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3282.147585] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3282.147587] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3282.147588] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3282.147590] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3282.147591] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3282.147592] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3282.147593] Call Trace: [ 3282.147594] [ 3282.147595] ? show_regs.cold.9+0x22/0x2f [ 3282.147596] ? nmi_cpu_backtrace.cold.8+0x98/0xa2 [ 3282.147597] ? nmi_cpu_backtrace_handler+0x15/0x40 [ 3282.147598] ? nmi_handle+0xbd/0x210 [ 3282.147598] ? default_do_nmi+0x4b/0x1b0 [ 3282.147599] ? do_nmi+0x280/0x380 [ 3282.147600] ? end_repeat_nmi+0x16/0x1f [ 3282.147601] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3282.147602] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3282.147602] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3282.147603] [ 3282.147604] do_raw_spin_lock+0x177/0x190 [ 3282.147605] _raw_spin_lock+0x21/0x30 [ 3282.147605] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3282.147606] ? do_raw_spin_unlock+0x75/0x190 [ 3282.147607] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3282.147608] ? do_wait_intr+0xf0/0xf0 [ 3282.147609] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3282.147610] mdc_reint+0x2d/0x110 [mdc] [ 3282.147611] mdc_unlink+0x658/0x870 [mdc] [ 3282.147612] lmv_unlink+0x34d/0xbd0 [lmv] [ 3282.147613] ll_unlink+0x24d/0x7f0 [lustre] [ 3282.147614] vfs_unlink+0x202/0x330 [ 3282.147615] do_unlinkat+0x2fa/0x460 [ 3282.147616] __x64_sys_unlink+0x37/0x50 [ 3282.147617] do_syscall_64+0xc1/0x440 [ 3282.147618] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3282.147619] RIP: 0033:0x7fe02c11910b [ 3282.147620] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3282.147622] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3282.147624] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3282.147625] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3282.147626] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3282.147628] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3282.147629] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3282.149045] NMI backtrace for cpu 1 [ 3282.528511] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3282.535969] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3282.546341] Call Trace: [ 3282.547713] [ 3282.548620] ? dump_stack+0xbb/0x10e [ 3282.554418] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3282.556398] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3282.564476] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3282.569142] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3282.572329] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3282.574892] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3282.577064] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3282.579997] ? update_process_times+0x6d/0xc0 [ 3282.582387] ? tick_sched_handle.isra.18+0x27/0x90 [ 3282.584392] ? tick_sched_timer+0x53/0xb0 [ 3282.585953] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3282.587950] ? tick_sched_handle.isra.18+0x90/0x90 [ 3282.590111] ? hrtimer_interrupt+0x128/0x2c0 [ 3282.591955] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3282.593925] ? apic_timer_interrupt+0xf/0x20 [ 3282.595968] [ 3282.596984] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3282.599299] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3282.601665] ? do_raw_spin_lock+0x177/0x190 [ 3282.603573] ? _raw_spin_lock+0x21/0x30 [ 3282.605400] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3282.607666] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3282.609937] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3282.615867] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3282.622520] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3282.636897] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3282.639668] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3282.644925] ? __next_timer_interrupt+0x160/0x160 [ 3282.649575] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3282.651616] ? do_wait_intr+0xf0/0xf0 [ 3282.659901] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3282.664948] ? kthread+0x1d1/0x200 [ 3282.669910] ? set_kthread_struct+0x70/0x70 [ 3282.675137] ? ret_from_fork+0x1f/0x30 [ 3288.064082] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3288.071651] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3288.116028] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3288.122802] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3288.127892] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3288.134341] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3288.137472] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3288.151984] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3288.155886] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3288.160146] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3288.162198] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3288.179114] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3288.185026] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3288.192071] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3288.198282] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3288.203323] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3288.207944] Call Trace: [ 3288.209877] [ 3288.210619] ? show_regs.cold.9+0x22/0x2f [ 3288.216782] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3288.220030] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3288.228224] ? watchdog+0x40/0x40 [ 3288.232043] ? hrtimer_interrupt+0x128/0x2c0 [ 3288.233580] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3288.236925] ? apic_timer_interrupt+0xf/0x20 [ 3288.239906] [ 3288.242988] ? native_safe_halt+0x2e/0x50 [ 3288.247627] kvm_wait+0xcd/0xe0 [ 3288.249286] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3288.254567] do_raw_spin_lock+0x177/0x190 [ 3288.258646] _raw_spin_lock+0x21/0x30 [ 3288.259944] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3288.268648] process_one_work+0x2c8/0x700 [ 3288.270788] worker_thread+0x296/0x6e0 [ 3288.272806] ? rescuer_thread+0x570/0x570 [ 3288.276064] kthread+0x1d1/0x200 [ 3288.282893] ? set_kthread_struct+0x70/0x70 [ 3288.291994] ret_from_fork+0x1f/0x30 [ 3308.057191] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3308.060027] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3308.060982] Modules linked in: [ 3308.063876] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix [ 3308.065347] lustre(O) [ 3308.065350] serio_raw libata dm_mirror dm_region_hash [ 3308.082343] osp(O) [ 3308.083474] dm_log [ 3308.086531] ofd(O) [ 3308.088240] dm_mod sha512_ssse3 sha512_generic [ 3308.088254] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3308.088260] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3308.088264] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3308.088277] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3308.088282] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3308.088287] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3308.088289] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3308.088291] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3308.088293] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3308.088295] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3308.088298] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3308.088301] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3308.088303] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3308.088312] Call Trace: [ 3308.088316] [ 3308.088322] ? show_regs.cold.9+0x22/0x2f [ 3308.088332] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3308.088338] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3308.088342] ? watchdog+0x40/0x40 [ 3308.088347] ? hrtimer_interrupt+0x128/0x2c0 [ 3308.088350] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3308.088357] ? apic_timer_interrupt+0xf/0x20 [ 3308.088362] [ 3308.088365] ? native_safe_halt+0x2e/0x50 [ 3308.088369] kvm_wait+0xcd/0xe0 [ 3308.088377] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3308.088385] do_raw_spin_lock+0x177/0x190 [ 3308.088389] _raw_spin_lock+0x21/0x30 [ 3308.088395] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3308.088648] obd_mod_free+0x46/0x140 [ptlrpc] [ 3308.088928] mdc_commit_open+0x91/0xb0 [mdc] [ 3308.088961] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3308.089227] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3308.089423] after_reply+0xe2c/0x1780 [ptlrpc] [ 3308.089615] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3308.090018] ? __next_timer_interrupt+0x160/0x160 [ 3308.090028] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3308.090232] ? do_wait_intr+0xf0/0xf0 [ 3308.090241] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3308.090598] kthread+0x1d1/0x200 [ 3308.090603] ? set_kthread_struct+0x70/0x70 [ 3308.090607] ret_from_fork+0x1f/0x30 [ 3308.258444] lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3308.284952] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3308.289634] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3308.293927] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3308.299890] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3308.312447] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3308.316639] RAX: 0000000000007b10 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3308.319434] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3308.323245] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3308.326931] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3308.330938] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3308.337596] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3308.342553] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3308.344590] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3308.349618] Call Trace: [ 3308.353897] [ 3308.355087] ? show_regs.cold.9+0x22/0x2f [ 3308.357324] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3308.359989] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3308.362891] ? watchdog+0x40/0x40 [ 3308.364287] ? hrtimer_interrupt+0x128/0x2c0 [ 3308.369989] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3308.373101] ? apic_timer_interrupt+0xf/0x20 [ 3308.374830] [ 3308.376067] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3308.378794] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3308.385166] do_raw_spin_lock+0x177/0x190 [ 3308.389112] _raw_spin_lock+0x21/0x30 [ 3308.390359] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3308.395594] ? do_raw_spin_unlock+0x75/0x190 [ 3308.397555] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3308.400177] ? do_wait_intr+0xf0/0xf0 [ 3308.402132] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3308.406051] mdc_reint+0x2d/0x110 [mdc] [ 3308.408181] mdc_unlink+0x658/0x870 [mdc] [ 3308.411618] lmv_unlink+0x34d/0xbd0 [lmv] [ 3308.413252] ll_unlink+0x24d/0x7f0 [lustre] [ 3308.415023] vfs_unlink+0x202/0x330 [ 3308.416365] do_unlinkat+0x2fa/0x460 [ 3308.418193] __x64_sys_unlink+0x37/0x50 [ 3308.420531] do_syscall_64+0xc1/0x440 [ 3308.423324] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3308.427948] RIP: 0033:0x7fe02c11910b [ 3308.434015] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3308.442681] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3308.446340] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3308.449314] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3308.452240] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3308.454505] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3308.458163] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3316.064048] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3316.069703] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3316.091991] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3316.097238] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3316.101391] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3316.104073] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3316.110321] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3316.119742] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3316.125539] RAX: 0000000000006a4d RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3316.128254] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3316.131625] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3316.140454] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3316.146121] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3316.149062] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3316.152117] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3316.154471] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3316.157229] Call Trace: [ 3316.158093] [ 3316.158794] ? show_regs.cold.9+0x22/0x2f [ 3316.160355] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3316.161919] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3316.165946] ? watchdog+0x40/0x40 [ 3316.170242] ? hrtimer_interrupt+0x128/0x2c0 [ 3316.171619] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3316.176194] ? apic_timer_interrupt+0xf/0x20 [ 3316.178458] [ 3316.179208] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3316.184210] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3316.186928] do_raw_spin_lock+0x177/0x190 [ 3316.188333] _raw_spin_lock+0x21/0x30 [ 3316.190017] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3316.193518] process_one_work+0x2c8/0x700 [ 3316.195103] worker_thread+0x296/0x6e0 [ 3316.196699] ? rescuer_thread+0x570/0x570 [ 3316.198433] kthread+0x1d1/0x200 [ 3316.199844] ? set_kthread_struct+0x70/0x70 [ 3316.201670] ret_from_fork+0x1f/0x30 [ 3336.057680] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3336.060016] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3336.067469] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3336.073508] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3336.073594] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3336.073602] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3336.073605] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3336.073620] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3336.073625] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3336.073630] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3336.073632] RDX: 0000000000003cc3 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3336.073634] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3336.073636] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3336.073639] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3336.073641] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3336.073645] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3336.073647] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3336.073656] Call Trace: [ 3336.073661] [ 3336.073667] ? show_regs.cold.9+0x22/0x2f [ 3336.073677] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3336.073683] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3336.073687] ? watchdog+0x40/0x40 [ 3336.073691] ? hrtimer_interrupt+0x128/0x2c0 [ 3336.073695] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3336.073704] ? apic_timer_interrupt+0xf/0x20 [ 3336.073708] [ 3336.073710] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3336.073716] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3336.073720] do_raw_spin_lock+0x177/0x190 [ 3336.073725] _raw_spin_lock+0x21/0x30 [ 3336.073731] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3336.113985] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3336.131811] obd_mod_free+0x46/0x140 [ptlrpc] [ 3336.132176] mdc_commit_open+0x91/0xb0 [mdc] [ 3336.132216] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3336.132414] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3336.132808] after_reply+0xe2c/0x1780 [ptlrpc] [ 3336.133004] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3336.138557] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3336.138566] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3336.138580] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3336.138584] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 3336.138590] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3336.138593] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3336.138595] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3336.138597] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3336.138600] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3336.138603] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3336.138606] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3336.138608] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3336.138617] Call Trace: [ 3336.138621] [ 3336.138626] ? show_regs.cold.9+0x22/0x2f [ 3336.138635] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3336.138641] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3336.138645] ? watchdog+0x40/0x40 [ 3336.138649] ? hrtimer_interrupt+0x128/0x2c0 [ 3336.138653] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3336.138659] ? apic_timer_interrupt+0xf/0x20 [ 3336.138664] [ 3336.138666] ? native_safe_halt+0x2e/0x50 [ 3336.138670] kvm_wait+0xcd/0xe0 [ 3336.138676] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3336.138683] do_raw_spin_lock+0x177/0x190 [ 3336.138687] _raw_spin_lock+0x21/0x30 [ 3336.138693] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3336.138981] ? do_raw_spin_unlock+0x75/0x190 [ 3336.138986] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3336.139198] ? do_wait_intr+0xf0/0xf0 [ 3336.139206] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3336.139408] mdc_reint+0x2d/0x110 [mdc] [ 3336.139444] mdc_unlink+0x658/0x870 [mdc] [ 3336.139472] lmv_unlink+0x34d/0xbd0 [lmv] [ 3336.139502] ll_unlink+0x24d/0x7f0 [lustre] [ 3336.139640] vfs_unlink+0x202/0x330 [ 3336.139647] do_unlinkat+0x2fa/0x460 [ 3336.139652] __x64_sys_unlink+0x37/0x50 [ 3336.139655] do_syscall_64+0xc1/0x440 [ 3336.139661] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3336.139667] RIP: 0033:0x7fe02c11910b [ 3336.139673] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3336.139676] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3336.139681] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3336.139684] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3336.139686] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3336.139688] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3336.139690] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3336.495431] ? __next_timer_interrupt+0x160/0x160 [ 3336.498360] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3336.500792] ? do_wait_intr+0xf0/0xf0 [ 3336.502027] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3336.504148] kthread+0x1d1/0x200 [ 3336.505400] ? set_kthread_struct+0x70/0x70 [ 3336.507240] ret_from_fork+0x1f/0x30 [ 3344.066214] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3344.071799] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3344.107086] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3344.112576] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3344.115939] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3344.118668] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3344.120801] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3344.125964] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3344.128306] RAX: 0000000000007d8d RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3344.131469] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3344.133751] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3344.135304] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3344.137920] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3344.139360] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3344.141033] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3344.142465] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3344.144349] Call Trace: [ 3344.144925] [ 3344.145349] ? show_regs.cold.9+0x22/0x2f [ 3344.146250] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3344.147317] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3344.148986] ? watchdog+0x40/0x40 [ 3344.150161] ? hrtimer_interrupt+0x128/0x2c0 [ 3344.151961] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3344.153476] ? apic_timer_interrupt+0xf/0x20 [ 3344.154862] [ 3344.156578] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3344.159131] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3344.161241] do_raw_spin_lock+0x177/0x190 [ 3344.162678] _raw_spin_lock+0x21/0x30 [ 3344.163592] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3344.165357] process_one_work+0x2c8/0x700 [ 3344.166857] worker_thread+0x296/0x6e0 [ 3344.168151] ? rescuer_thread+0x570/0x570 [ 3344.169639] kthread+0x1d1/0x200 [ 3344.170571] ? set_kthread_struct+0x70/0x70 [ 3344.171769] ret_from_fork+0x1f/0x30 [ 3364.057024] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3364.059477] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3364.060025] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3364.083072] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3364.086663] Modules linked in: lustre(O) [ 3364.094397] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3364.094407] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3364.094421] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3364.094425] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3364.094431] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3364.094433] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3364.094435] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3364.094438] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3364.094440] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3364.094443] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3364.094447] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3364.094449] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3364.094458] Call Trace: [ 3364.094463] [ 3364.094469] ? show_regs.cold.9+0x22/0x2f [ 3364.094478] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3364.094485] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3364.094491] ? watchdog+0x40/0x40 [ 3364.094495] ? hrtimer_interrupt+0x128/0x2c0 [ 3364.094500] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3364.094507] ? apic_timer_interrupt+0xf/0x20 [ 3364.094512] [ 3364.094515] ? native_safe_halt+0x2e/0x50 [ 3364.094520] kvm_wait+0xcd/0xe0 [ 3364.094527] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3364.094535] do_raw_spin_lock+0x177/0x190 [ 3364.094539] _raw_spin_lock+0x21/0x30 [ 3364.094546] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3364.096366] osp(O) [ 3364.101074] ? do_raw_spin_unlock+0x75/0x190 [ 3364.102878] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3364.102948] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3364.102954] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3364.102956] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3364.102968] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3364.102973] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3364.102977] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3364.102980] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3364.102982] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3364.102984] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3364.102986] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3364.102989] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3364.102992] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3364.102994] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3364.103002] Call Trace: [ 3364.103016] [ 3364.103021] ? show_regs.cold.9+0x22/0x2f [ 3364.103030] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3364.103037] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3364.103042] ? watchdog+0x40/0x40 [ 3364.103045] ? hrtimer_interrupt+0x128/0x2c0 [ 3364.103049] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3364.103104] ? apic_timer_interrupt+0xf/0x20 [ 3364.103109] [ 3364.103112] ? native_safe_halt+0x2e/0x50 [ 3364.103116] kvm_wait+0xcd/0xe0 [ 3364.103122] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3364.103128] do_raw_spin_lock+0x177/0x190 [ 3364.103132] _raw_spin_lock+0x21/0x30 [ 3364.103136] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3364.103400] obd_mod_free+0x46/0x140 [ptlrpc] [ 3364.103588] mdc_commit_open+0x91/0xb0 [mdc] [ 3364.103614] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3364.103798] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3364.103987] after_reply+0xe2c/0x1780 [ptlrpc] [ 3364.104357] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3364.104546] ? __next_timer_interrupt+0x160/0x160 [ 3364.104554] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3364.104817] ? do_wait_intr+0xf0/0xf0 [ 3364.104826] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3364.114474] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3364.118168] kthread+0x1d1/0x200 [ 3364.120202] ? do_wait_intr+0xf0/0xf0 [ 3364.122487] ? set_kthread_struct+0x70/0x70 [ 3364.124686] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3364.126991] ret_from_fork+0x1f/0x30 [ 3364.129460] mdc_reint+0x2d/0x110 [mdc] [ 3364.349351] mdc_unlink+0x658/0x870 [mdc] [ 3364.353504] lmv_unlink+0x34d/0xbd0 [lmv] [ 3364.357744] ll_unlink+0x24d/0x7f0 [lustre] [ 3364.367417] vfs_unlink+0x202/0x330 [ 3364.368674] do_unlinkat+0x2fa/0x460 [ 3364.369818] __x64_sys_unlink+0x37/0x50 [ 3364.376738] do_syscall_64+0xc1/0x440 [ 3364.379461] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3364.385290] RIP: 0033:0x7fe02c11910b [ 3364.390761] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3364.414195] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3364.417887] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3364.421443] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3364.425356] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3364.428416] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3364.431635] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3372.066036] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3372.073510] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3372.107020] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3372.116412] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3372.125514] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3372.131727] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3372.134929] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3372.145572] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3372.149458] RAX: 0000000000006625 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3372.153353] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3372.156303] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3372.158487] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3372.162543] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3372.166112] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3372.170087] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3372.174405] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3372.178418] Call Trace: [ 3372.179832] [ 3372.180459] ? show_regs.cold.9+0x22/0x2f [ 3372.182063] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3372.184081] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3372.188091] ? watchdog+0x40/0x40 [ 3372.190244] ? hrtimer_interrupt+0x128/0x2c0 [ 3372.195146] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3372.197507] ? apic_timer_interrupt+0xf/0x20 [ 3372.199432] [ 3372.200609] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3372.205185] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3372.208283] do_raw_spin_lock+0x177/0x190 [ 3372.210319] _raw_spin_lock+0x21/0x30 [ 3372.212312] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3372.215895] process_one_work+0x2c8/0x700 [ 3372.222308] worker_thread+0x296/0x6e0 [ 3372.225545] ? rescuer_thread+0x570/0x570 [ 3372.227807] kthread+0x1d1/0x200 [ 3372.229777] ? set_kthread_struct+0x70/0x70 [ 3372.231653] ret_from_fork+0x1f/0x30 [ 3392.057034] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3392.060146] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3392.060554] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3392.060620] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3392.060626] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3392.060629] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3392.060643] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3392.060647] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3392.060651] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3392.060654] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3392.060655] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3392.060657] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3392.060659] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3392.060661] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3392.060664] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3392.060666] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3392.060675] Call Trace: [ 3392.060680] [ 3392.060686] ? show_regs.cold.9+0x22/0x2f [ 3392.060695] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3392.060701] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3392.060705] ? watchdog+0x40/0x40 [ 3392.060710] ? hrtimer_interrupt+0x128/0x2c0 [ 3392.060713] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3392.060719] ? apic_timer_interrupt+0xf/0x20 [ 3392.060724] [ 3392.060727] ? native_safe_halt+0x2e/0x50 [ 3392.060730] kvm_wait+0xcd/0xe0 [ 3392.060737] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3392.060743] do_raw_spin_lock+0x177/0x190 [ 3392.060748] _raw_spin_lock+0x21/0x30 [ 3392.060753] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3392.074379] Modules linked in: lustre(O) [ 3392.117965] ? do_raw_spin_unlock+0x75/0x190 [ 3392.117985] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3392.118259] ? do_wait_intr+0xf0/0xf0 [ 3392.118267] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3392.118670] mdc_reint+0x2d/0x110 [mdc] [ 3392.118708] mdc_unlink+0x658/0x870 [mdc] [ 3392.118733] lmv_unlink+0x34d/0xbd0 [lmv] [ 3392.118761] ll_unlink+0x24d/0x7f0 [lustre] [ 3392.118869] vfs_unlink+0x202/0x330 [ 3392.118879] do_unlinkat+0x2fa/0x460 [ 3392.118884] __x64_sys_unlink+0x37/0x50 [ 3392.118888] do_syscall_64+0xc1/0x440 [ 3392.118896] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3392.118903] RIP: 0033:0x7fe02c11910b [ 3392.118912] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3392.118916] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3392.118921] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3392.118923] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3392.118925] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3392.118927] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3392.118928] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3392.406319] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3392.465992] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3392.478363] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3392.485734] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3392.493901] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3392.514410] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3392.523426] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3392.532151] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3392.541940] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3392.550886] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3392.561404] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3392.573274] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3392.587821] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3392.604691] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3392.609104] Call Trace: [ 3392.612033] [ 3392.615045] ? show_regs.cold.9+0x22/0x2f [ 3392.618938] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3392.621418] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3392.624228] ? watchdog+0x40/0x40 [ 3392.625888] ? hrtimer_interrupt+0x128/0x2c0 [ 3392.628130] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3392.631419] ? apic_timer_interrupt+0xf/0x20 [ 3392.637360] [ 3392.640050] ? native_safe_halt+0x2e/0x50 [ 3392.644608] kvm_wait+0xcd/0xe0 [ 3392.652334] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3392.654830] do_raw_spin_lock+0x177/0x190 [ 3392.659804] _raw_spin_lock+0x21/0x30 [ 3392.671882] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3392.675335] obd_mod_free+0x46/0x140 [ptlrpc] [ 3392.681270] mdc_commit_open+0x91/0xb0 [mdc] [ 3392.690894] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3392.697003] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3392.702376] after_reply+0xe2c/0x1780 [ptlrpc] [ 3392.715135] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3392.717083] ? __next_timer_interrupt+0x160/0x160 [ 3392.744430] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3392.763356] ? do_wait_intr+0xf0/0xf0 [ 3392.764603] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3392.776363] kthread+0x1d1/0x200 [ 3392.782946] ? set_kthread_struct+0x70/0x70 [ 3392.785810] ret_from_fork+0x1f/0x30 [ 3400.064991] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3400.073238] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3400.111125] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3400.115058] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3400.117745] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3400.121368] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3400.123256] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3400.128048] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3400.141261] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3400.146575] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3400.153563] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3400.161961] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3400.167269] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3400.175071] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3400.181271] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3400.184247] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3400.187325] Call Trace: [ 3400.190105] [ 3400.191421] ? show_regs.cold.9+0x22/0x2f [ 3400.194514] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3400.197608] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3400.200387] ? watchdog+0x40/0x40 [ 3400.203137] ? hrtimer_interrupt+0x128/0x2c0 [ 3400.207160] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3400.214543] ? apic_timer_interrupt+0xf/0x20 [ 3400.224280] [ 3400.227583] ? native_safe_halt+0x2e/0x50 [ 3400.229434] kvm_wait+0xcd/0xe0 [ 3400.232540] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3400.234560] do_raw_spin_lock+0x177/0x190 [ 3400.236091] _raw_spin_lock+0x21/0x30 [ 3400.239257] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3400.245475] process_one_work+0x2c8/0x700 [ 3400.248891] worker_thread+0x296/0x6e0 [ 3400.250640] ? rescuer_thread+0x570/0x570 [ 3400.254288] kthread+0x1d1/0x200 [ 3400.255582] ? set_kthread_struct+0x70/0x70 [ 3400.257242] ret_from_fork+0x1f/0x30 [ 3420.057027] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3420.060093] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3420.060185] Modules linked in: [ 3420.064367] Modules linked in: lustre(O) osp(O) [ 3420.066282] lustre(O) osp(O) [ 3420.068666] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3420.068756] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3420.068763] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3420.068766] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3420.068833] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3420.068838] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3420.068843] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3420.068845] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3420.068848] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3420.068850] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3420.068853] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3420.068856] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3420.068858] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3420.068860] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3420.068869] Call Trace: [ 3420.068886] [ 3420.068893] ? show_regs.cold.9+0x22/0x2f [ 3420.068903] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3420.068915] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3420.068920] ? watchdog+0x40/0x40 [ 3420.068924] ? hrtimer_interrupt+0x128/0x2c0 [ 3420.068929] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3420.068935] ? apic_timer_interrupt+0xf/0x20 [ 3420.068940] [ 3420.068943] ? native_safe_halt+0x2e/0x50 [ 3420.068947] kvm_wait+0xcd/0xe0 [ 3420.068957] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3420.068964] do_raw_spin_lock+0x177/0x190 [ 3420.068969] _raw_spin_lock+0x21/0x30 [ 3420.068973] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3420.069423] obd_mod_free+0x46/0x140 [ptlrpc] [ 3420.072872] ofd(O) [ 3420.099494] mdc_commit_open+0x91/0xb0 [mdc] [ 3420.110291] lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3420.110360] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3420.110367] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3420.110370] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3420.110383] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3420.110387] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3420.110392] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3420.110395] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3420.110397] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3420.110400] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3420.110402] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3420.110405] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3420.110407] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3420.110409] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3420.110417] Call Trace: [ 3420.110423] [ 3420.110428] ? show_regs.cold.9+0x22/0x2f [ 3420.110439] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3420.110446] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3420.110450] ? watchdog+0x40/0x40 [ 3420.110455] ? hrtimer_interrupt+0x128/0x2c0 [ 3420.110460] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3420.110466] ? apic_timer_interrupt+0xf/0x20 [ 3420.110471] [ 3420.110474] ? native_safe_halt+0x2e/0x50 [ 3420.110477] kvm_wait+0xcd/0xe0 [ 3420.110484] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3420.110491] do_raw_spin_lock+0x177/0x190 [ 3420.110496] _raw_spin_lock+0x21/0x30 [ 3420.110501] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3420.110848] ? do_raw_spin_unlock+0x75/0x190 [ 3420.110854] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3420.111047] ? do_wait_intr+0xf0/0xf0 [ 3420.111055] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3420.111259] mdc_reint+0x2d/0x110 [mdc] [ 3420.111299] mdc_unlink+0x658/0x870 [mdc] [ 3420.111327] lmv_unlink+0x34d/0xbd0 [lmv] [ 3420.111357] ll_unlink+0x24d/0x7f0 [lustre] [ 3420.111469] vfs_unlink+0x202/0x330 [ 3420.111477] do_unlinkat+0x2fa/0x460 [ 3420.111482] __x64_sys_unlink+0x37/0x50 [ 3420.111486] do_syscall_64+0xc1/0x440 [ 3420.111493] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3420.111500] RIP: 0033:0x7fe02c11910b [ 3420.111506] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3420.111510] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3420.111514] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3420.111517] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3420.111519] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3420.111521] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3420.111523] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3420.509045] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3420.513468] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3420.517722] after_reply+0xe2c/0x1780 [ptlrpc] [ 3420.520811] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3420.526829] ? __next_timer_interrupt+0x160/0x160 [ 3420.530849] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3420.534873] ? do_wait_intr+0xf0/0xf0 [ 3420.537376] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3420.540573] kthread+0x1d1/0x200 [ 3420.542025] ? set_kthread_struct+0x70/0x70 [ 3420.543870] ret_from_fork+0x1f/0x30 [ 3428.067664] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3428.075095] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3428.114812] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3428.124279] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3428.130115] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3428.133146] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3428.137037] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3428.147433] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3428.158912] RAX: 0000000000007df4 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3428.162840] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3428.165828] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3428.167994] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3428.170103] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3428.172214] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3428.174929] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3428.176838] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3428.179881] Call Trace: [ 3428.181227] [ 3428.182120] ? show_regs.cold.9+0x22/0x2f [ 3428.183731] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3428.186018] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3428.190998] ? watchdog+0x40/0x40 [ 3428.193557] ? hrtimer_interrupt+0x128/0x2c0 [ 3428.196704] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3428.201096] ? apic_timer_interrupt+0xf/0x20 [ 3428.202540] [ 3428.207220] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3428.209300] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3428.214190] do_raw_spin_lock+0x177/0x190 [ 3428.215356] _raw_spin_lock+0x21/0x30 [ 3428.217276] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3428.219297] process_one_work+0x2c8/0x700 [ 3428.224007] worker_thread+0x296/0x6e0 [ 3428.225480] ? rescuer_thread+0x570/0x570 [ 3428.226865] kthread+0x1d1/0x200 [ 3428.228153] ? set_kthread_struct+0x70/0x70 [ 3428.229767] ret_from_fork+0x1f/0x30 [ 3448.057018] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3448.060013] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3448.061252] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3448.082625] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3448.085999] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) [ 3448.090755] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3448.090769] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3448.090787] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3448.090791] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3448.090797] RAX: 000000000000401f RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3448.090799] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3448.090801] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3448.090803] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3448.090805] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3448.090811] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3448.090815] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3448.090820] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3448.090827] Call Trace: [ 3448.090833] [ 3448.090838] ? show_regs.cold.9+0x22/0x2f [ 3448.090849] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3448.090862] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3448.090867] ? watchdog+0x40/0x40 [ 3448.090871] ? hrtimer_interrupt+0x128/0x2c0 [ 3448.090874] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3448.090882] ? apic_timer_interrupt+0xf/0x20 [ 3448.090887] [ 3448.090889] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3448.097767] lov(O) [ 3448.101489] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3448.103953] osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common [ 3448.109122] do_raw_spin_lock+0x177/0x190 [ 3448.111278] sb_edac [ 3448.113395] _raw_spin_lock+0x21/0x30 [ 3448.113412] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3448.116254] rapl [ 3448.118331] ? do_raw_spin_unlock+0x75/0x190 [ 3448.120565] pcspkr [ 3448.122902] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3448.125495] i2c_piix4 [ 3448.127249] ? do_wait_intr+0xf0/0xf0 [ 3448.129496] squashfs [ 3448.130265] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3448.130963] ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3448.130987] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3448.130993] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3448.130996] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3448.132103] mdc_reint+0x2d/0x110 [mdc] [ 3448.133328] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3448.135458] mdc_unlink+0x658/0x870 [mdc] [ 3448.135504] lmv_unlink+0x34d/0xbd0 [lmv] [ 3448.135534] ll_unlink+0x24d/0x7f0 [lustre] [ 3448.136508] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 [ 3448.138305] vfs_unlink+0x202/0x330 [ 3448.139934] ORIG_RAX: ffffffffffffff13 [ 3448.139941] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3448.139944] RDX: 0000000000000a2c RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3448.139946] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3448.139948] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3448.139950] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3448.139954] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3448.139956] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3448.139959] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3448.139968] Call Trace: [ 3448.139973] [ 3448.141125] do_unlinkat+0x2fa/0x460 [ 3448.142034] ? show_regs.cold.9+0x22/0x2f [ 3448.143307] __x64_sys_unlink+0x37/0x50 [ 3448.144142] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3448.145590] do_syscall_64+0xc1/0x440 [ 3448.145608] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3448.145616] RIP: 0033:0x7fe02c11910b [ 3448.145627] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3448.145632] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3448.145638] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3448.145640] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3448.145642] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3448.145644] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3448.145646] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3448.252169] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3448.253568] ? watchdog+0x40/0x40 [ 3448.254781] ? hrtimer_interrupt+0x128/0x2c0 [ 3448.256053] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3448.257790] ? apic_timer_interrupt+0xf/0x20 [ 3448.259097] [ 3448.259641] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3448.261448] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3448.263173] do_raw_spin_lock+0x177/0x190 [ 3448.264425] _raw_spin_lock+0x21/0x30 [ 3448.265517] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3448.267432] obd_mod_free+0x46/0x140 [ptlrpc] [ 3448.269156] mdc_commit_open+0x91/0xb0 [mdc] [ 3448.270715] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3448.272793] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3448.274891] after_reply+0xe2c/0x1780 [ptlrpc] [ 3448.280401] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3448.284940] ? __next_timer_interrupt+0x160/0x160 [ 3448.287921] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3448.289476] ? do_wait_intr+0xf0/0xf0 [ 3448.290737] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3448.292639] kthread+0x1d1/0x200 [ 3448.293764] ? set_kthread_struct+0x70/0x70 [ 3448.295193] ret_from_fork+0x1f/0x30 [ 3456.064125] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3456.067461] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3456.137340] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3456.156306] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3456.165080] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3456.170118] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3456.174774] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3456.183892] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3456.186906] RAX: 0000000000002d99 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3456.190924] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3456.200148] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3456.211090] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3456.214686] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3456.222677] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3456.226724] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3456.229173] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3456.232740] Call Trace: [ 3456.233838] [ 3456.238746] ? show_regs.cold.9+0x22/0x2f [ 3456.241124] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3456.243869] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3456.248144] ? watchdog+0x40/0x40 [ 3456.249862] ? hrtimer_interrupt+0x128/0x2c0 [ 3456.254506] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3456.255862] ? apic_timer_interrupt+0xf/0x20 [ 3456.258730] [ 3456.259797] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3456.262634] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3456.265150] do_raw_spin_lock+0x177/0x190 [ 3456.268083] _raw_spin_lock+0x21/0x30 [ 3456.270628] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3456.274162] process_one_work+0x2c8/0x700 [ 3456.277416] worker_thread+0x296/0x6e0 [ 3456.287163] ? rescuer_thread+0x570/0x570 [ 3456.296022] kthread+0x1d1/0x200 [ 3456.301791] ? set_kthread_struct+0x70/0x70 [ 3456.309377] ret_from_fork+0x1f/0x30 [ 3462.127278] rcu: INFO: rcu_sched self-detected stall on CPU [ 3462.129441] rcu: 1-....: (853636 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=249812 [ 3462.133674] (t=1320035 jiffies g=278549 q=8202) [ 3462.135618] Sending NMI from CPU 1 to CPUs 0: [ 3462.141334] NMI backtrace for cpu 0 [ 3462.141341] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3462.141343] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3462.141345] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3462.141346] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3462.141348] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000212 [ 3462.141351] RAX: 000000000000329b RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3462.141353] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3462.141537] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3462.141539] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3462.141540] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3462.141541] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3462.141543] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3462.141544] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3462.141545] Call Trace: [ 3462.141546] [ 3462.141547] ? show_regs.cold.9+0x22/0x2f [ 3462.141548] ? nmi_cpu_backtrace.cold.8+0x98/0xa2 [ 3462.141549] ? nmi_cpu_backtrace_handler+0x15/0x40 [ 3462.141550] ? nmi_handle+0xbd/0x210 [ 3462.141551] ? default_do_nmi+0x4b/0x1b0 [ 3462.141553] ? do_nmi+0x280/0x380 [ 3462.141554] ? end_repeat_nmi+0x16/0x1f [ 3462.141555] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3462.141556] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3462.141557] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3462.141559] [ 3462.141559] do_raw_spin_lock+0x177/0x190 [ 3462.141561] _raw_spin_lock+0x21/0x30 [ 3462.141562] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3462.141563] ? do_raw_spin_unlock+0x75/0x190 [ 3462.141564] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3462.141565] ? do_wait_intr+0xf0/0xf0 [ 3462.141566] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3462.141567] mdc_reint+0x2d/0x110 [mdc] [ 3462.141568] mdc_unlink+0x658/0x870 [mdc] [ 3462.141569] lmv_unlink+0x34d/0xbd0 [lmv] [ 3462.141571] ll_unlink+0x24d/0x7f0 [lustre] [ 3462.141572] vfs_unlink+0x202/0x330 [ 3462.141573] do_unlinkat+0x2fa/0x460 [ 3462.141574] __x64_sys_unlink+0x37/0x50 [ 3462.141575] do_syscall_64+0xc1/0x440 [ 3462.141576] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3462.141577] RIP: 0033:0x7fe02c11910b [ 3462.141579] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3462.141581] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3462.141584] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3462.141586] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3462.141587] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3462.141589] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3462.141590] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3462.141797] NMI backtrace for cpu 1 [ 3462.328652] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3462.333737] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3462.337530] Call Trace: [ 3462.338568] [ 3462.339988] ? dump_stack+0xbb/0x10e [ 3462.342470] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3462.349748] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3462.352058] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3462.354406] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3462.356588] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3462.358251] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3462.359907] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3462.361884] ? update_process_times+0x6d/0xc0 [ 3462.363236] ? tick_sched_handle.isra.18+0x27/0x90 [ 3462.364949] ? tick_sched_timer+0x53/0xb0 [ 3462.366235] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3462.367607] ? tick_sched_handle.isra.18+0x90/0x90 [ 3462.369140] ? hrtimer_interrupt+0x128/0x2c0 [ 3462.370545] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3462.378264] ? apic_timer_interrupt+0xf/0x20 [ 3462.381037] [ 3462.382901] ? __pv_queued_spin_lock_slowpath+0x185/0x6d0 [ 3462.385898] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3462.387285] ? do_raw_spin_lock+0x177/0x190 [ 3462.388369] ? _raw_spin_lock+0x21/0x30 [ 3462.390505] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3462.393194] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3462.396135] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3462.398257] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3462.404140] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3462.407780] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3462.410516] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3462.412765] ? __next_timer_interrupt+0x160/0x160 [ 3462.414772] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3462.417012] ? do_wait_intr+0xf0/0xf0 [ 3462.418966] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3462.421067] ? kthread+0x1d1/0x200 [ 3462.422685] ? set_kthread_struct+0x70/0x70 [ 3462.424726] ? ret_from_fork+0x1f/0x30 [ 3484.064587] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3484.068112] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3484.090363] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3484.094694] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3484.098667] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3484.101545] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3484.103690] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3484.111041] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3484.114835] RAX: 000000000000152f RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3484.118306] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3484.122271] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3484.125756] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3484.128814] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3484.131701] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3484.137032] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3484.139880] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3484.143795] Call Trace: [ 3484.145271] [ 3484.146297] ? show_regs.cold.9+0x22/0x2f [ 3484.148526] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3484.151041] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3484.152985] ? watchdog+0x40/0x40 [ 3484.154230] ? hrtimer_interrupt+0x128/0x2c0 [ 3484.156723] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3484.158950] ? apic_timer_interrupt+0xf/0x20 [ 3484.161326] [ 3484.162624] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3484.165869] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3484.168890] do_raw_spin_lock+0x177/0x190 [ 3484.171392] _raw_spin_lock+0x21/0x30 [ 3484.173356] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3484.178378] process_one_work+0x2c8/0x700 [ 3484.180092] worker_thread+0x296/0x6e0 [ 3484.182313] ? rescuer_thread+0x570/0x570 [ 3484.184497] kthread+0x1d1/0x200 [ 3484.186344] ? set_kthread_struct+0x70/0x70 [ 3484.189390] ret_from_fork+0x1f/0x30 [ 3488.060022] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3488.060812] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3488.073748] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3488.073824] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3488.073831] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3488.073834] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3488.073848] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3488.073853] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3488.073857] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3488.073860] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3488.073862] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3488.073865] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3488.073868] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3488.073871] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3488.073873] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3488.073876] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3488.073885] Call Trace: [ 3488.073891] [ 3488.073896] ? show_regs.cold.9+0x22/0x2f [ 3488.073907] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3488.073913] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3488.073916] ? watchdog+0x40/0x40 [ 3488.073921] ? hrtimer_interrupt+0x128/0x2c0 [ 3488.073924] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3488.073930] ? apic_timer_interrupt+0xf/0x20 [ 3488.073934] [ 3488.073937] ? native_safe_halt+0x2e/0x50 [ 3488.073941] kvm_wait+0xcd/0xe0 [ 3488.073948] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3488.073956] do_raw_spin_lock+0x177/0x190 [ 3488.073961] _raw_spin_lock+0x21/0x30 [ 3488.073966] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3488.078627] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3488.078710] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3488.078717] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3488.078720] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3488.078734] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3488.078738] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3488.078743] RAX: 000000000000650c RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3488.078745] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3488.078748] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3488.078750] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3488.078752] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3488.078755] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3488.078758] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3488.078760] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3488.078770] Call Trace: [ 3488.078776] [ 3488.078781] ? show_regs.cold.9+0x22/0x2f [ 3488.078790] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3488.078797] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3488.078802] ? watchdog+0x40/0x40 [ 3488.078808] ? hrtimer_interrupt+0x128/0x2c0 [ 3488.078812] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3488.078820] ? apic_timer_interrupt+0xf/0x20 [ 3488.078825] [ 3488.078828] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3488.078832] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3488.078836] do_raw_spin_lock+0x177/0x190 [ 3488.078840] _raw_spin_lock+0x21/0x30 [ 3488.078845] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3488.079124] ? do_raw_spin_unlock+0x75/0x190 [ 3488.079131] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3488.079330] ? do_wait_intr+0xf0/0xf0 [ 3488.079339] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3488.079537] mdc_reint+0x2d/0x110 [mdc] [ 3488.079577] mdc_unlink+0x658/0x870 [mdc] [ 3488.079601] lmv_unlink+0x34d/0xbd0 [lmv] [ 3488.079631] ll_unlink+0x24d/0x7f0 [lustre] [ 3488.079747] vfs_unlink+0x202/0x330 [ 3488.079756] do_unlinkat+0x2fa/0x460 [ 3488.079761] __x64_sys_unlink+0x37/0x50 [ 3488.079765] do_syscall_64+0xc1/0x440 [ 3488.079772] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3488.079778] RIP: 0033:0x7fe02c11910b [ 3488.079785] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3488.079789] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3488.079793] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3488.079796] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3488.079797] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3488.079799] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3488.079801] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3488.569365] obd_mod_free+0x46/0x140 [ptlrpc] [ 3488.572701] mdc_commit_open+0x91/0xb0 [mdc] [ 3488.575000] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3488.577933] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3488.580834] after_reply+0xe2c/0x1780 [ptlrpc] [ 3488.583921] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3488.586083] ? __next_timer_interrupt+0x160/0x160 [ 3488.590624] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3488.592357] ? do_wait_intr+0xf0/0xf0 [ 3488.594988] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3488.597023] kthread+0x1d1/0x200 [ 3488.599014] ? set_kthread_struct+0x70/0x70 [ 3488.601107] ret_from_fork+0x1f/0x30 [ 3512.066587] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3512.068932] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3512.108304] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3512.124636] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3512.133572] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3512.142407] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3512.143827] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3512.163614] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3512.174843] RAX: 0000000000007f0f RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3512.179760] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3512.184542] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3512.192032] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3512.196008] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3512.200389] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3512.206525] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3512.208936] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3512.212224] Call Trace: [ 3512.213391] [ 3512.214154] ? show_regs.cold.9+0x22/0x2f [ 3512.215278] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3512.221235] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3512.225246] ? watchdog+0x40/0x40 [ 3512.227799] ? hrtimer_interrupt+0x128/0x2c0 [ 3512.230992] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3512.235073] ? apic_timer_interrupt+0xf/0x20 [ 3512.236971] [ 3512.237657] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3512.240673] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3512.243180] do_raw_spin_lock+0x177/0x190 [ 3512.245720] _raw_spin_lock+0x21/0x30 [ 3512.247445] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3512.250705] process_one_work+0x2c8/0x700 [ 3512.252783] worker_thread+0x296/0x6e0 [ 3512.254730] ? rescuer_thread+0x570/0x570 [ 3512.255972] kthread+0x1d1/0x200 [ 3512.257056] ? set_kthread_struct+0x70/0x70 [ 3512.258520] ret_from_fork+0x1f/0x30 [ 3516.057129] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3516.059709] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3516.060009] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3516.077184] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3516.080224] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3516.080299] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3516.080306] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3516.080308] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3516.080323] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3516.080327] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3516.080331] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3516.080334] RDX: 0000000000001ed8 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3516.080336] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3516.080339] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3516.080346] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3516.080349] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3516.080352] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3516.080355] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3516.080364] Call Trace: [ 3516.080369] [ 3516.080375] ? show_regs.cold.9+0x22/0x2f [ 3516.080384] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3516.080390] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3516.080454] ? watchdog+0x40/0x40 [ 3516.080459] ? hrtimer_interrupt+0x128/0x2c0 [ 3516.080463] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3516.080471] ? apic_timer_interrupt+0xf/0x20 [ 3516.080476] [ 3516.080478] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3516.080483] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3516.080487] do_raw_spin_lock+0x177/0x190 [ 3516.080492] _raw_spin_lock+0x21/0x30 [ 3516.080498] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3516.080857] obd_mod_free+0x46/0x140 [ptlrpc] [ 3516.085890] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3516.085904] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3516.085921] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3516.085925] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3516.085931] RAX: 0000000000006aa9 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3516.085933] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3516.085935] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3516.085937] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3516.085939] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3516.085942] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3516.085944] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3516.085946] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3516.085954] Call Trace: [ 3516.085960] [ 3516.085966] ? show_regs.cold.9+0x22/0x2f [ 3516.085975] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3516.085982] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3516.085986] ? watchdog+0x40/0x40 [ 3516.085990] ? hrtimer_interrupt+0x128/0x2c0 [ 3516.085994] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3516.086002] ? apic_timer_interrupt+0xf/0x20 [ 3516.086005] [ 3516.086008] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3516.086027] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3516.086032] do_raw_spin_lock+0x177/0x190 [ 3516.086037] _raw_spin_lock+0x21/0x30 [ 3516.086043] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3516.086334] ? do_raw_spin_unlock+0x75/0x190 [ 3516.086339] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3516.086749] ? do_wait_intr+0xf0/0xf0 [ 3516.086758] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3516.087194] mdc_reint+0x2d/0x110 [mdc] [ 3516.087313] mdc_unlink+0x658/0x870 [mdc] [ 3516.087342] lmv_unlink+0x34d/0xbd0 [lmv] [ 3516.087371] ll_unlink+0x24d/0x7f0 [lustre] [ 3516.087573] vfs_unlink+0x202/0x330 [ 3516.087581] do_unlinkat+0x2fa/0x460 [ 3516.087586] __x64_sys_unlink+0x37/0x50 [ 3516.087590] do_syscall_64+0xc1/0x440 [ 3516.087596] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3516.087679] RIP: 0033:0x7fe02c11910b [ 3516.087696] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3516.087700] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3516.087705] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3516.087708] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3516.087710] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3516.087712] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3516.087714] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3516.324366] mdc_commit_open+0x91/0xb0 [mdc] [ 3516.327216] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3516.332167] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3516.335789] after_reply+0xe2c/0x1780 [ptlrpc] [ 3516.337767] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3516.342914] ? __next_timer_interrupt+0x160/0x160 [ 3516.347601] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3516.351818] ? do_wait_intr+0xf0/0xf0 [ 3516.354704] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3516.358856] kthread+0x1d1/0x200 [ 3516.362171] ? set_kthread_struct+0x70/0x70 [ 3516.363246] ret_from_fork+0x1f/0x30 [ 3540.064337] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3540.067508] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3540.115452] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3540.123346] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3540.140613] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3540.147553] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3540.155800] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3540.177914] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3540.194617] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3540.201772] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3540.207692] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3540.216042] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3540.218213] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3540.234746] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3540.244128] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3540.252692] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3540.263960] Call Trace: [ 3540.271643] [ 3540.272839] ? show_regs.cold.9+0x22/0x2f [ 3540.276075] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3540.282920] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3540.291468] ? watchdog+0x40/0x40 [ 3540.293310] ? hrtimer_interrupt+0x128/0x2c0 [ 3540.295136] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3540.297406] ? apic_timer_interrupt+0xf/0x20 [ 3540.299158] [ 3540.299824] ? native_safe_halt+0x2e/0x50 [ 3540.301265] kvm_wait+0xcd/0xe0 [ 3540.302244] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3540.304145] do_raw_spin_lock+0x177/0x190 [ 3540.306639] _raw_spin_lock+0x21/0x30 [ 3540.307876] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3540.314947] process_one_work+0x2c8/0x700 [ 3540.317067] worker_thread+0x296/0x6e0 [ 3540.319231] ? rescuer_thread+0x570/0x570 [ 3540.321123] kthread+0x1d1/0x200 [ 3540.326402] ? set_kthread_struct+0x70/0x70 [ 3540.333311] ret_from_fork+0x1f/0x30 [ 3544.060011] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3544.061907] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3544.066337] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3544.066418] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3544.066426] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3544.066436] RIP: 0010:__pv_queued_spin_lock_slowpath+0x17c/0x6d0 [ 3544.066452] Code: 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 48 83 05 5c d6 dc 02 01 <83> ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 75 e6 48 83 05 2c d6 dc [ 3544.066456] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3544.066461] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3544.066463] RDX: 0000000000004b07 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3544.066465] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3544.066467] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3544.066469] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3544.066472] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3544.066475] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3544.066477] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3544.066486] Call Trace: [ 3544.066493] [ 3544.066499] ? show_regs.cold.9+0x22/0x2f [ 3544.066509] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3544.066516] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3544.066527] ? watchdog+0x40/0x40 [ 3544.066533] ? hrtimer_interrupt+0x128/0x2c0 [ 3544.066537] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3544.066546] ? apic_timer_interrupt+0xf/0x20 [ 3544.066550] [ 3544.066557] ? __pv_queued_spin_lock_slowpath+0x17c/0x6d0 [ 3544.066562] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3544.066568] do_raw_spin_lock+0x177/0x190 [ 3544.066572] _raw_spin_lock+0x21/0x30 [ 3544.066578] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3544.067229] obd_mod_free+0x46/0x140 [ptlrpc] [ 3544.067574] mdc_commit_open+0x91/0xb0 [mdc] [ 3544.067614] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3544.067832] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3544.071394] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3544.071608] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3544.071617] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3544.071620] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3544.071634] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3544.071698] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3544.071704] RAX: 00000000000079e1 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3544.071707] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3544.071710] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3544.071712] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3544.071714] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3544.071718] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3544.071724] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3544.071727] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3544.071734] Call Trace: [ 3544.071739] [ 3544.071745] ? show_regs.cold.9+0x22/0x2f [ 3544.071755] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3544.071761] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3544.071765] ? watchdog+0x40/0x40 [ 3544.071768] ? hrtimer_interrupt+0x128/0x2c0 [ 3544.071772] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3544.071780] ? apic_timer_interrupt+0xf/0x20 [ 3544.071785] [ 3544.071787] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3544.071792] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3544.071797] do_raw_spin_lock+0x177/0x190 [ 3544.071802] _raw_spin_lock+0x21/0x30 [ 3544.071807] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3544.072321] ? do_raw_spin_unlock+0x75/0x190 [ 3544.072332] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3544.072757] ? do_wait_intr+0xf0/0xf0 [ 3544.072764] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3544.099192] after_reply+0xe2c/0x1780 [ptlrpc] [ 3544.100100] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3544.100305] ? __next_timer_interrupt+0x160/0x160 [ 3544.100316] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3544.100555] ? do_wait_intr+0xf0/0xf0 [ 3544.100564] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3544.101113] kthread+0x1d1/0x200 [ 3544.101119] ? set_kthread_struct+0x70/0x70 [ 3544.101123] ret_from_fork+0x1f/0x30 [ 3544.369774] mdc_reint+0x2d/0x110 [mdc] [ 3544.371299] mdc_unlink+0x658/0x870 [mdc] [ 3544.373208] lmv_unlink+0x34d/0xbd0 [lmv] [ 3544.375421] ll_unlink+0x24d/0x7f0 [lustre] [ 3544.377376] vfs_unlink+0x202/0x330 [ 3544.378894] do_unlinkat+0x2fa/0x460 [ 3544.380685] __x64_sys_unlink+0x37/0x50 [ 3544.382451] do_syscall_64+0xc1/0x440 [ 3544.384306] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3544.386714] RIP: 0033:0x7fe02c11910b [ 3544.388360] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3544.397226] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3544.400753] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3544.404162] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3544.407450] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3544.410662] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3544.414177] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3568.064335] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3568.070891] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3568.092670] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3568.096916] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3568.100932] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3568.104527] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3568.107027] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3568.115860] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3568.119438] RAX: 0000000000001a1d RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3568.122812] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3568.126521] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3568.129832] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3568.133127] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3568.137078] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3568.141302] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3568.144195] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3568.147456] Call Trace: [ 3568.148732] [ 3568.149808] ? show_regs.cold.9+0x22/0x2f [ 3568.151689] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3568.153987] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3568.156053] ? watchdog+0x40/0x40 [ 3568.157654] ? hrtimer_interrupt+0x128/0x2c0 [ 3568.159536] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3568.162250] ? apic_timer_interrupt+0xf/0x20 [ 3568.164709] [ 3568.165339] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3568.168332] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3568.170617] do_raw_spin_lock+0x177/0x190 [ 3568.172967] _raw_spin_lock+0x21/0x30 [ 3568.175675] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3568.178135] process_one_work+0x2c8/0x700 [ 3568.180189] worker_thread+0x296/0x6e0 [ 3568.182288] ? rescuer_thread+0x570/0x570 [ 3568.183982] kthread+0x1d1/0x200 [ 3568.185695] ? set_kthread_struct+0x70/0x70 [ 3568.187759] ret_from_fork+0x1f/0x30 [ 3572.057024] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3572.060026] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3572.060292] Modules linked in: lustre(O) osp(O) ofd(O) [ 3572.069363] Modules linked in: [ 3572.069364] lod(O) mdt(O) mdd(O) mgs(O) [ 3572.073833] lustre(O) [ 3572.074651] lquota(O) [ 3572.077194] osp(O) [ 3572.078118] lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3572.078162] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3572.078168] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3572.078171] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3572.078225] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3572.078229] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3572.078234] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3572.078236] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3572.078238] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3572.078240] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3572.078242] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3572.078246] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3572.078248] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3572.078250] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3572.078258] Call Trace: [ 3572.078275] [ 3572.078281] ? show_regs.cold.9+0x22/0x2f [ 3572.078289] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3572.078294] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3572.078297] ? watchdog+0x40/0x40 [ 3572.078300] ? hrtimer_interrupt+0x128/0x2c0 [ 3572.078302] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3572.078307] ? apic_timer_interrupt+0xf/0x20 [ 3572.078311] [ 3572.078313] ? native_safe_halt+0x2e/0x50 [ 3572.078317] kvm_wait+0xcd/0xe0 [ 3572.078324] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3572.078331] do_raw_spin_lock+0x177/0x190 [ 3572.078335] _raw_spin_lock+0x21/0x30 [ 3572.078339] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3572.078643] ? do_raw_spin_unlock+0x75/0x190 [ 3572.078647] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3572.081597] ofd(O) [ 3572.082630] ? do_wait_intr+0xf0/0xf0 [ 3572.109738] lod(O) [ 3572.114603] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3572.120831] mdt(O) [ 3572.122691] mdc_reint+0x2d/0x110 [mdc] [ 3572.133062] mdd(O) [ 3572.136196] mdc_unlink+0x658/0x870 [mdc] [ 3572.140890] mgs(O) [ 3572.144348] lmv_unlink+0x34d/0xbd0 [lmv] [ 3572.148009] lquota(O) [ 3572.150869] ll_unlink+0x24d/0x7f0 [lustre] [ 3572.155560] lfsck(O) [ 3572.159904] vfs_unlink+0x202/0x330 [ 3572.163229] obdecho(O) [ 3572.166404] do_unlinkat+0x2fa/0x460 [ 3572.166433] __x64_sys_unlink+0x37/0x50 [ 3572.166437] do_syscall_64+0xc1/0x440 [ 3572.166452] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3572.166474] RIP: 0033:0x7fe02c11910b [ 3572.166482] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3572.166486] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3572.166492] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3572.166495] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3572.166497] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3572.166499] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3572.166501] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3572.294630] mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3572.316082] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3572.326652] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3572.333753] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3572.336948] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3572.358965] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3572.364588] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3572.369128] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3572.373082] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3572.376574] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3572.381050] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3572.385248] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3572.395176] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3572.398952] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3572.402180] Call Trace: [ 3572.403158] [ 3572.404106] ? show_regs.cold.9+0x22/0x2f [ 3572.405972] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3572.407941] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3572.409919] ? watchdog+0x40/0x40 [ 3572.411336] ? hrtimer_interrupt+0x128/0x2c0 [ 3572.413767] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3572.415644] ? apic_timer_interrupt+0xf/0x20 [ 3572.418108] [ 3572.419044] ? native_safe_halt+0x2e/0x50 [ 3572.420851] kvm_wait+0xcd/0xe0 [ 3572.422006] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3572.424422] do_raw_spin_lock+0x177/0x190 [ 3572.426021] _raw_spin_lock+0x21/0x30 [ 3572.427598] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3572.429929] obd_mod_free+0x46/0x140 [ptlrpc] [ 3572.432008] mdc_commit_open+0x91/0xb0 [mdc] [ 3572.433918] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3572.436348] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3572.438770] after_reply+0xe2c/0x1780 [ptlrpc] [ 3572.441335] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3572.443776] ? __next_timer_interrupt+0x160/0x160 [ 3572.445873] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3572.448216] ? do_wait_intr+0xf0/0xf0 [ 3572.449821] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3572.452286] kthread+0x1d1/0x200 [ 3572.453527] ? set_kthread_struct+0x70/0x70 [ 3572.455424] ret_from_fork+0x1f/0x30 [ 3596.066185] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3596.076869] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3596.147841] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3596.158668] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3596.168195] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3596.172108] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3596.174956] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3596.191335] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3596.195189] RAX: 0000000000006b0c RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3596.198287] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3596.201227] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3596.204210] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3596.207200] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3596.211302] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3596.214915] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3596.217328] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3596.220474] Call Trace: [ 3596.221515] [ 3596.222439] ? show_regs.cold.9+0x22/0x2f [ 3596.224133] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3596.226304] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3596.228118] ? watchdog+0x40/0x40 [ 3596.229667] ? hrtimer_interrupt+0x128/0x2c0 [ 3596.231254] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3596.233334] ? apic_timer_interrupt+0xf/0x20 [ 3596.234929] [ 3596.235684] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3596.237852] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3596.240528] do_raw_spin_lock+0x177/0x190 [ 3596.242644] _raw_spin_lock+0x21/0x30 [ 3596.244984] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3596.248203] process_one_work+0x2c8/0x700 [ 3596.249865] worker_thread+0x296/0x6e0 [ 3596.251413] ? rescuer_thread+0x570/0x570 [ 3596.253132] kthread+0x1d1/0x200 [ 3596.254426] ? set_kthread_struct+0x70/0x70 [ 3596.256259] ret_from_fork+0x1f/0x30 [ 3600.057166] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3600.060028] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3600.061302] Modules linked in: [ 3600.065449] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3600.065523] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3600.065531] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3600.065534] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3600.065559] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3600.065563] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3600.065568] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3600.065570] RDX: 00000000000032bd RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3600.065572] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3600.065575] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3600.065577] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3600.065580] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3600.065583] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3600.065585] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3600.065594] Call Trace: [ 3600.065600] [ 3600.065606] ? show_regs.cold.9+0x22/0x2f [ 3600.065615] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3600.065622] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3600.065626] ? watchdog+0x40/0x40 [ 3600.065630] ? hrtimer_interrupt+0x128/0x2c0 [ 3600.065634] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3600.065642] ? apic_timer_interrupt+0xf/0x20 [ 3600.065646] [ 3600.065649] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3600.065653] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3600.065657] do_raw_spin_lock+0x177/0x190 [ 3600.065662] _raw_spin_lock+0x21/0x30 [ 3600.065668] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3600.066250] obd_mod_free+0x46/0x140 [ptlrpc] [ 3600.066447] mdc_commit_open+0x91/0xb0 [mdc] [ 3600.066487] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3600.066698] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3600.070049] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3600.070120] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3600.070127] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3600.070130] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3600.070143] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3600.070147] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3600.070152] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3600.070154] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3600.070156] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3600.070158] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3600.070160] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3600.070164] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3600.070167] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3600.070169] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3600.070177] Call Trace: [ 3600.070183] [ 3600.070188] ? show_regs.cold.9+0x22/0x2f [ 3600.070198] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3600.070205] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3600.070208] ? watchdog+0x40/0x40 [ 3600.070213] ? hrtimer_interrupt+0x128/0x2c0 [ 3600.070217] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3600.070224] ? apic_timer_interrupt+0xf/0x20 [ 3600.070229] [ 3600.070232] ? native_safe_halt+0x2e/0x50 [ 3600.070235] kvm_wait+0xcd/0xe0 [ 3600.070243] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3600.070251] do_raw_spin_lock+0x177/0x190 [ 3600.070255] _raw_spin_lock+0x21/0x30 [ 3600.070260] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3600.070506] ? do_raw_spin_unlock+0x75/0x190 [ 3600.070513] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3600.099831] after_reply+0xe2c/0x1780 [ptlrpc] [ 3600.100108] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3600.100301] ? __next_timer_interrupt+0x160/0x160 [ 3600.100312] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3600.100514] ? do_wait_intr+0xf0/0xf0 [ 3600.100524] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3600.100812] kthread+0x1d1/0x200 [ 3600.100818] ? set_kthread_struct+0x70/0x70 [ 3600.100822] ret_from_fork+0x1f/0x30 [ 3600.403837] ? do_wait_intr+0xf0/0xf0 [ 3600.406662] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3600.416328] mdc_reint+0x2d/0x110 [mdc] [ 3600.417735] mdc_unlink+0x658/0x870 [mdc] [ 3600.422943] lmv_unlink+0x34d/0xbd0 [lmv] [ 3600.424475] ll_unlink+0x24d/0x7f0 [lustre] [ 3600.426238] vfs_unlink+0x202/0x330 [ 3600.427255] do_unlinkat+0x2fa/0x460 [ 3600.428143] __x64_sys_unlink+0x37/0x50 [ 3600.429076] do_syscall_64+0xc1/0x440 [ 3600.429875] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3600.431514] RIP: 0033:0x7fe02c11910b [ 3600.432915] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3600.440242] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3600.442686] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3600.444939] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3600.447039] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3600.449835] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3600.452554] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3624.066496] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3624.068920] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3624.089838] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3624.095179] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3624.098457] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3624.101329] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3624.103571] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3624.111728] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3624.115453] RAX: 000000000000780b RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3624.120341] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3624.123613] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3624.127267] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3624.130502] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3624.133463] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3624.136813] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3624.141980] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3624.148189] Call Trace: [ 3624.149056] [ 3624.150550] ? show_regs.cold.9+0x22/0x2f [ 3624.152179] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3624.154302] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3624.156342] ? watchdog+0x40/0x40 [ 3624.157755] ? hrtimer_interrupt+0x128/0x2c0 [ 3624.159350] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3624.161267] ? apic_timer_interrupt+0xf/0x20 [ 3624.162994] [ 3624.163893] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3624.166848] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3624.168930] do_raw_spin_lock+0x177/0x190 [ 3624.170743] _raw_spin_lock+0x21/0x30 [ 3624.172572] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3624.175824] process_one_work+0x2c8/0x700 [ 3624.177923] worker_thread+0x296/0x6e0 [ 3624.179410] ? rescuer_thread+0x570/0x570 [ 3624.180993] kthread+0x1d1/0x200 [ 3624.184476] ? set_kthread_struct+0x70/0x70 [ 3624.186247] ret_from_fork+0x1f/0x30 [ 3628.057027] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3628.060019] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3628.063269] Modules linked in: [ 3628.067111] Modules linked in: [ 3628.067115] lustre(O) osp(O) [ 3628.068624] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3628.068839] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3628.068847] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3628.068850] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3628.068864] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3628.068868] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3628.068872] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3628.068874] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3628.068876] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3628.068878] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3628.068881] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3628.068884] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3628.068887] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3628.068889] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3628.068897] Call Trace: [ 3628.068902] [ 3628.068908] ? show_regs.cold.9+0x22/0x2f [ 3628.068918] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3628.068924] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3628.068929] ? watchdog+0x40/0x40 [ 3628.068933] ? hrtimer_interrupt+0x128/0x2c0 [ 3628.068937] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3628.068949] ? apic_timer_interrupt+0xf/0x20 [ 3628.068956] [ 3628.068958] ? native_safe_halt+0x2e/0x50 [ 3628.068962] kvm_wait+0xcd/0xe0 [ 3628.068969] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3628.068977] do_raw_spin_lock+0x177/0x190 [ 3628.068981] _raw_spin_lock+0x21/0x30 [ 3628.068985] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3628.069919] obd_mod_free+0x46/0x140 [ptlrpc] [ 3628.070136] mdc_commit_open+0x91/0xb0 [mdc] [ 3628.073970] ofd(O) [ 3628.076221] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3628.108985] lod(O) [ 3628.116243] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3628.116521] after_reply+0xe2c/0x1780 [ptlrpc] [ 3628.116717] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3628.116914] ? __next_timer_interrupt+0x160/0x160 [ 3628.116924] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3628.121392] mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3628.121470] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3628.121477] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3628.121479] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3628.121491] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3628.121495] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3628.121500] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3628.121502] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3628.121504] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3628.121506] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3628.121509] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3628.121512] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3628.121514] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3628.121516] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3628.121525] Call Trace: [ 3628.121532] [ 3628.121537] ? show_regs.cold.9+0x22/0x2f [ 3628.121549] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3628.121555] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3628.121560] ? watchdog+0x40/0x40 [ 3628.121565] ? hrtimer_interrupt+0x128/0x2c0 [ 3628.121569] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3628.121576] ? apic_timer_interrupt+0xf/0x20 [ 3628.121580] [ 3628.121583] ? native_safe_halt+0x2e/0x50 [ 3628.121587] kvm_wait+0xcd/0xe0 [ 3628.121594] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3628.121601] do_raw_spin_lock+0x177/0x190 [ 3628.121606] _raw_spin_lock+0x21/0x30 [ 3628.121611] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3628.122073] ? do_raw_spin_unlock+0x75/0x190 [ 3628.122078] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3628.122294] ? do_wait_intr+0xf0/0xf0 [ 3628.122301] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3628.122560] mdc_reint+0x2d/0x110 [mdc] [ 3628.122599] mdc_unlink+0x658/0x870 [mdc] [ 3628.122737] lmv_unlink+0x34d/0xbd0 [lmv] [ 3628.122773] ll_unlink+0x24d/0x7f0 [lustre] [ 3628.122906] vfs_unlink+0x202/0x330 [ 3628.122913] do_unlinkat+0x2fa/0x460 [ 3628.122918] __x64_sys_unlink+0x37/0x50 [ 3628.122921] do_syscall_64+0xc1/0x440 [ 3628.122928] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3628.122934] RIP: 0033:0x7fe02c11910b [ 3628.122942] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3628.122945] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3628.122950] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3628.122952] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3628.122954] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3628.122957] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3628.122959] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3628.486239] ? do_wait_intr+0xf0/0xf0 [ 3628.487655] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3628.489740] kthread+0x1d1/0x200 [ 3628.490980] ? set_kthread_struct+0x70/0x70 [ 3628.492536] ret_from_fork+0x1f/0x30 [ 3642.130852] rcu: INFO: rcu_sched self-detected stall on CPU [ 3642.132621] rcu: 1-....: (977932 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=285099 [ 3642.141571] (t=1500038 jiffies g=278549 q=8876) [ 3642.144181] Sending NMI from CPU 1 to CPUs 0: [ 3642.146900] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 [ 3642.147861] NMI backtrace for cpu 1 [ 3642.153847] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3642.161843] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3642.168412] Call Trace: [ 3642.169748] [ 3642.171961] ? dump_stack+0xbb/0x10e [ 3642.174093] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3642.176160] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3642.177687] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3642.179216] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3642.181406] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3642.183235] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3642.184692] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3642.186092] ? update_process_times+0x6d/0xc0 [ 3642.187632] ? tick_sched_handle.isra.18+0x27/0x90 [ 3642.189548] ? tick_sched_timer+0x53/0xb0 [ 3642.190758] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3642.192647] ? tick_sched_handle.isra.18+0x90/0x90 [ 3642.194182] ? hrtimer_interrupt+0x128/0x2c0 [ 3642.195377] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3642.196688] ? apic_timer_interrupt+0xf/0x20 [ 3642.197793] [ 3642.198510] ? native_safe_halt+0x2e/0x50 [ 3642.199715] ? kvm_wait+0xcd/0xe0 [ 3642.200662] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3642.202235] ? do_raw_spin_lock+0x177/0x190 [ 3642.203341] ? _raw_spin_lock+0x21/0x30 [ 3642.204598] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3642.206950] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3642.209617] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3642.211705] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3642.214656] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3642.219338] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3642.221992] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3642.225398] ? __next_timer_interrupt+0x160/0x160 [ 3642.227630] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3642.230116] ? do_wait_intr+0xf0/0xf0 [ 3642.233750] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3642.238939] ? kthread+0x1d1/0x200 [ 3642.243985] ? set_kthread_struct+0x70/0x70 [ 3642.246956] ? ret_from_fork+0x1f/0x30 [ 3652.066751] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3652.076479] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3652.126774] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3652.137555] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3652.144672] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3652.152436] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3652.155784] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3652.172124] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3652.176110] RAX: 0000000000004f62 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3652.179892] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3652.182711] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3652.186308] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3652.190812] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3652.194826] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3652.203254] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3652.206182] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3652.209938] Call Trace: [ 3652.214237] [ 3652.215561] ? show_regs.cold.9+0x22/0x2f [ 3652.217378] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3652.220467] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3652.223715] ? watchdog+0x40/0x40 [ 3652.226957] ? hrtimer_interrupt+0x128/0x2c0 [ 3652.231420] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3652.237152] ? apic_timer_interrupt+0xf/0x20 [ 3652.238817] [ 3652.239815] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3652.245986] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3652.253398] do_raw_spin_lock+0x177/0x190 [ 3652.255030] _raw_spin_lock+0x21/0x30 [ 3652.256933] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3652.260295] process_one_work+0x2c8/0x700 [ 3652.262373] worker_thread+0x296/0x6e0 [ 3652.264817] ? rescuer_thread+0x570/0x570 [ 3652.270909] kthread+0x1d1/0x200 [ 3652.276256] ? set_kthread_struct+0x70/0x70 [ 3652.280790] ret_from_fork+0x1f/0x30 [ 3668.057705] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3668.060514] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3668.060660] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3668.086272] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3668.088466] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3668.088540] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3668.088547] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3668.088549] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3668.088562] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3668.088565] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3668.088569] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3668.088572] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3668.088574] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3668.088576] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3668.088578] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3668.088581] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3668.088584] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3668.088586] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3668.088593] Call Trace: [ 3668.088597] [ 3668.088602] ? show_regs.cold.9+0x22/0x2f [ 3668.088610] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3668.088617] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3668.088620] ? watchdog+0x40/0x40 [ 3668.088624] ? hrtimer_interrupt+0x128/0x2c0 [ 3668.088628] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3668.088634] ? apic_timer_interrupt+0xf/0x20 [ 3668.088638] [ 3668.088640] ? native_safe_halt+0x2e/0x50 [ 3668.088644] kvm_wait+0xcd/0xe0 [ 3668.088650] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3668.088656] do_raw_spin_lock+0x177/0x190 [ 3668.088660] _raw_spin_lock+0x21/0x30 [ 3668.088665] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3668.099908] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3668.099916] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3668.099928] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3668.099931] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3668.099936] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3668.099938] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3668.099942] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3668.099944] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3668.099946] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3668.099948] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3668.099950] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3668.099952] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3668.099957] Call Trace: [ 3668.099962] [ 3668.099966] ? show_regs.cold.9+0x22/0x2f [ 3668.099973] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3668.099978] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3668.099981] ? watchdog+0x40/0x40 [ 3668.099984] ? hrtimer_interrupt+0x128/0x2c0 [ 3668.099986] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3668.099992] ? apic_timer_interrupt+0xf/0x20 [ 3668.099995] [ 3668.099997] ? native_safe_halt+0x2e/0x50 [ 3668.099999] kvm_wait+0xcd/0xe0 [ 3668.100006] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3668.100011] do_raw_spin_lock+0x177/0x190 [ 3668.100013] _raw_spin_lock+0x21/0x30 [ 3668.100018] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3668.100274] ? do_raw_spin_unlock+0x75/0x190 [ 3668.100279] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3668.100483] ? do_wait_intr+0xf0/0xf0 [ 3668.100490] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3668.100684] mdc_reint+0x2d/0x110 [mdc] [ 3668.101988] mdc_unlink+0x658/0x870 [mdc] [ 3668.118584] obd_mod_free+0x46/0x140 [ptlrpc] [ 3668.123235] lmv_unlink+0x34d/0xbd0 [lmv] [ 3668.123279] ll_unlink+0x24d/0x7f0 [lustre] [ 3668.129036] mdc_commit_open+0x91/0xb0 [mdc] [ 3668.129094] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3668.129388] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3668.129588] after_reply+0xe2c/0x1780 [ptlrpc] [ 3668.129794] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3668.129978] ? __next_timer_interrupt+0x160/0x160 [ 3668.129985] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3668.130137] ? do_wait_intr+0xf0/0xf0 [ 3668.130146] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3668.130290] kthread+0x1d1/0x200 [ 3668.130294] ? set_kthread_struct+0x70/0x70 [ 3668.130297] ret_from_fork+0x1f/0x30 [ 3668.251608] vfs_unlink+0x202/0x330 [ 3668.252808] do_unlinkat+0x2fa/0x460 [ 3668.254126] __x64_sys_unlink+0x37/0x50 [ 3668.255321] do_syscall_64+0xc1/0x440 [ 3668.256499] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3668.258025] RIP: 0033:0x7fe02c11910b [ 3668.259079] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3668.264681] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3668.267009] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3668.269228] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3668.271031] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3668.273287] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3668.275629] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3680.067096] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3680.074657] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3680.128420] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3680.147152] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3680.156692] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3680.167482] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3680.170900] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3680.182669] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3680.188260] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3680.199962] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3680.205517] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3680.209302] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3680.212330] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3680.215263] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3680.218252] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3680.220155] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3680.224958] Call Trace: [ 3680.226041] [ 3680.229985] ? show_regs.cold.9+0x22/0x2f [ 3680.231463] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3680.234506] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3680.242170] ? watchdog+0x40/0x40 [ 3680.245651] ? hrtimer_interrupt+0x128/0x2c0 [ 3680.248196] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3680.251223] ? apic_timer_interrupt+0xf/0x20 [ 3680.253588] [ 3680.255783] ? native_safe_halt+0x2e/0x50 [ 3680.259661] kvm_wait+0xcd/0xe0 [ 3680.263040] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3680.268751] do_raw_spin_lock+0x177/0x190 [ 3680.270111] _raw_spin_lock+0x21/0x30 [ 3680.279120] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3680.288101] process_one_work+0x2c8/0x700 [ 3680.297193] worker_thread+0x296/0x6e0 [ 3680.303131] ? rescuer_thread+0x570/0x570 [ 3680.306033] kthread+0x1d1/0x200 [ 3680.309677] ? set_kthread_struct+0x70/0x70 [ 3680.316503] ret_from_fork+0x1f/0x30 [ 3696.057431] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3696.060126] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3696.066208] Modules linked in: [ 3696.070146] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3696.070226] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3696.070236] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3696.070239] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3696.070251] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3696.070255] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3696.070259] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3696.070262] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3696.070264] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3696.070266] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3696.070268] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3696.070272] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3696.070274] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3696.070277] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3696.070284] Call Trace: [ 3696.070295] [ 3696.070302] ? show_regs.cold.9+0x22/0x2f [ 3696.070311] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3696.070317] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3696.070321] ? watchdog+0x40/0x40 [ 3696.070325] ? hrtimer_interrupt+0x128/0x2c0 [ 3696.070340] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3696.070346] ? apic_timer_interrupt+0xf/0x20 [ 3696.070351] [ 3696.070354] ? native_safe_halt+0x2e/0x50 [ 3696.070358] kvm_wait+0xcd/0xe0 [ 3696.070366] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3696.070373] do_raw_spin_lock+0x177/0x190 [ 3696.070377] _raw_spin_lock+0x21/0x30 [ 3696.070383] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3696.070736] obd_mod_free+0x46/0x140 [ptlrpc] [ 3696.070883] mdc_commit_open+0x91/0xb0 [mdc] [ 3696.070929] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3696.071262] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3696.071505] after_reply+0xe2c/0x1780 [ptlrpc] [ 3696.071698] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3696.071897] ? __next_timer_interrupt+0x160/0x160 [ 3696.071905] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3696.072241] ? do_wait_intr+0xf0/0xf0 [ 3696.072255] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3696.072512] kthread+0x1d1/0x200 [ 3696.072517] ? set_kthread_struct+0x70/0x70 [ 3696.072521] ret_from_fork+0x1f/0x30 [ 3696.238301] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3696.266795] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3696.280169] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3696.284742] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3696.286516] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3696.296148] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3696.302231] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3696.308372] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3696.309990] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3696.314799] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3696.319235] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3696.322807] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3696.326319] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3696.328852] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3696.331916] Call Trace: [ 3696.333068] [ 3696.333991] ? show_regs.cold.9+0x22/0x2f [ 3696.335519] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3696.337107] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3696.338722] ? watchdog+0x40/0x40 [ 3696.339847] ? hrtimer_interrupt+0x128/0x2c0 [ 3696.342024] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3696.343758] ? apic_timer_interrupt+0xf/0x20 [ 3696.345577] [ 3696.346334] ? native_safe_halt+0x2e/0x50 [ 3696.347625] kvm_wait+0xcd/0xe0 [ 3696.348688] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3696.350508] do_raw_spin_lock+0x177/0x190 [ 3696.351657] _raw_spin_lock+0x21/0x30 [ 3696.352884] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3696.354772] ? do_raw_spin_unlock+0x75/0x190 [ 3696.356165] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3696.358192] ? do_wait_intr+0xf0/0xf0 [ 3696.359606] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3696.361680] mdc_reint+0x2d/0x110 [mdc] [ 3696.362937] mdc_unlink+0x658/0x870 [mdc] [ 3696.365612] lmv_unlink+0x34d/0xbd0 [lmv] [ 3696.368890] ll_unlink+0x24d/0x7f0 [lustre] [ 3696.374397] vfs_unlink+0x202/0x330 [ 3696.377118] do_unlinkat+0x2fa/0x460 [ 3696.379521] __x64_sys_unlink+0x37/0x50 [ 3696.383162] do_syscall_64+0xc1/0x440 [ 3696.385800] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3696.387525] RIP: 0033:0x7fe02c11910b [ 3696.391509] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3696.402024] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3696.407115] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3696.410080] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3696.419675] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3696.425504] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3696.429588] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3708.064060] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3708.068405] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3708.093061] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3708.099667] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3708.104877] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3708.107762] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3708.110440] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3708.118859] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3708.122879] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3708.126268] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3708.129756] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3708.134469] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3708.138835] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3708.142194] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3708.146612] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3708.149492] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3708.153939] Call Trace: [ 3708.155243] [ 3708.156358] ? show_regs.cold.9+0x22/0x2f [ 3708.158168] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3708.162267] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3708.166441] ? watchdog+0x40/0x40 [ 3708.167671] ? hrtimer_interrupt+0x128/0x2c0 [ 3708.169100] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3708.172470] ? apic_timer_interrupt+0xf/0x20 [ 3708.177620] [ 3708.179675] ? native_safe_halt+0x2e/0x50 [ 3708.182828] kvm_wait+0xcd/0xe0 [ 3708.185422] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3708.189117] do_raw_spin_lock+0x177/0x190 [ 3708.190823] _raw_spin_lock+0x21/0x30 [ 3708.192819] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3708.196538] process_one_work+0x2c8/0x700 [ 3708.197843] worker_thread+0x296/0x6e0 [ 3708.199147] ? rescuer_thread+0x570/0x570 [ 3708.200465] kthread+0x1d1/0x200 [ 3708.201439] ? set_kthread_struct+0x70/0x70 [ 3708.202834] ret_from_fork+0x1f/0x30 [ 3724.060416] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3724.066243] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3724.069742] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3724.093806] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3724.093820] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3724.093823] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3724.093838] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3724.093842] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3724.093848] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3724.093851] RDX: 000000000000679e RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3724.093859] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3724.093861] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3724.093863] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3724.093866] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3724.093870] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3724.093872] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3724.093880] Call Trace: [ 3724.093885] [ 3724.093891] ? show_regs.cold.9+0x22/0x2f [ 3724.093902] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3724.093909] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3724.093914] ? watchdog+0x40/0x40 [ 3724.093918] ? hrtimer_interrupt+0x128/0x2c0 [ 3724.093922] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3724.093930] ? apic_timer_interrupt+0xf/0x20 [ 3724.093935] [ 3724.093938] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3724.093943] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3724.093948] do_raw_spin_lock+0x177/0x190 [ 3724.093953] _raw_spin_lock+0x21/0x30 [ 3724.093960] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3724.094251] obd_mod_free+0x46/0x140 [ptlrpc] [ 3724.094612] mdc_commit_open+0x91/0xb0 [mdc] [ 3724.094652] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3724.094997] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3724.095197] after_reply+0xe2c/0x1780 [ptlrpc] [ 3724.098178] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3724.099082] ? __next_timer_interrupt+0x160/0x160 [ 3724.099094] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3724.099306] ? do_wait_intr+0xf0/0xf0 [ 3724.099316] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3724.099993] kthread+0x1d1/0x200 [ 3724.100001] ? set_kthread_struct+0x70/0x70 [ 3724.100004] ret_from_fork+0x1f/0x30 [ 3724.270063] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3724.300149] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3724.308975] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3724.312219] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3724.316657] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3724.329532] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3724.336390] RAX: 0000000000004125 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3724.342329] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3724.347469] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3724.351032] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3724.354596] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3724.360204] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3724.373114] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3724.377857] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3724.385896] Call Trace: [ 3724.387150] [ 3724.388874] ? show_regs.cold.9+0x22/0x2f [ 3724.392726] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3724.396655] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3724.399284] ? watchdog+0x40/0x40 [ 3724.400350] ? hrtimer_interrupt+0x128/0x2c0 [ 3724.403642] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3724.406252] ? apic_timer_interrupt+0xf/0x20 [ 3724.408392] [ 3724.409174] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3724.412871] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3724.418873] do_raw_spin_lock+0x177/0x190 [ 3724.422077] _raw_spin_lock+0x21/0x30 [ 3724.423526] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3724.426668] ? do_raw_spin_unlock+0x75/0x190 [ 3724.428971] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3724.431712] ? do_wait_intr+0xf0/0xf0 [ 3724.433220] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3724.434907] mdc_reint+0x2d/0x110 [mdc] [ 3724.438408] mdc_unlink+0x658/0x870 [mdc] [ 3724.444029] lmv_unlink+0x34d/0xbd0 [lmv] [ 3724.447203] ll_unlink+0x24d/0x7f0 [lustre] [ 3724.449259] vfs_unlink+0x202/0x330 [ 3724.450906] do_unlinkat+0x2fa/0x460 [ 3724.452300] __x64_sys_unlink+0x37/0x50 [ 3724.454068] do_syscall_64+0xc1/0x440 [ 3724.458156] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3724.462669] RIP: 0033:0x7fe02c11910b [ 3724.463874] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3724.471839] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3724.474966] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3724.477972] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3724.483342] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3724.487456] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3724.489574] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3736.071958] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3736.094668] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3736.174177] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3736.188364] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3736.200742] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3736.206364] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3736.216485] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3736.233479] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3736.239936] RAX: 0000000000007098 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3736.244313] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3736.255935] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3736.259119] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3736.262991] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3736.267430] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3736.273288] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3736.281865] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3736.295091] Call Trace: [ 3736.298919] [ 3736.301559] ? show_regs.cold.9+0x22/0x2f [ 3736.303324] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3736.310672] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3736.314109] ? watchdog+0x40/0x40 [ 3736.316178] ? hrtimer_interrupt+0x128/0x2c0 [ 3736.320748] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3736.331034] ? apic_timer_interrupt+0xf/0x20 [ 3736.336744] [ 3736.337498] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3736.347287] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3736.352269] do_raw_spin_lock+0x177/0x190 [ 3736.356077] _raw_spin_lock+0x21/0x30 [ 3736.368423] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3736.373267] process_one_work+0x2c8/0x700 [ 3736.379757] worker_thread+0x296/0x6e0 [ 3736.393590] ? rescuer_thread+0x570/0x570 [ 3736.396852] kthread+0x1d1/0x200 [ 3736.399067] ? set_kthread_struct+0x70/0x70 [ 3736.407245] ret_from_fork+0x1f/0x30 [ 3752.060125] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3752.062761] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3752.063598] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3752.063667] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3752.063673] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3752.063676] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3752.063690] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3752.063694] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3752.063699] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3752.063701] RDX: 0000000000007f85 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3752.063704] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3752.063706] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3752.063708] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3752.063711] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3752.063714] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3752.063716] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3752.063871] Call Trace: [ 3752.063877] [ 3752.063893] ? show_regs.cold.9+0x22/0x2f [ 3752.063904] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3752.063909] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3752.063913] ? watchdog+0x40/0x40 [ 3752.063917] ? hrtimer_interrupt+0x128/0x2c0 [ 3752.063920] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3752.063927] ? apic_timer_interrupt+0xf/0x20 [ 3752.063931] [ 3752.063934] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3752.063939] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3752.063943] do_raw_spin_lock+0x177/0x190 [ 3752.063948] _raw_spin_lock+0x21/0x30 [ 3752.063954] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3752.064667] obd_mod_free+0x46/0x140 [ptlrpc] [ 3752.069389] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3752.069462] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3752.069469] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3752.069472] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3752.069486] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3752.069490] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3752.069495] RAX: 0000000000003d81 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3752.069497] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3752.069499] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3752.069500] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3752.069503] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3752.069506] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3752.069509] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3752.069511] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3752.069520] Call Trace: [ 3752.069526] [ 3752.069532] ? show_regs.cold.9+0x22/0x2f [ 3752.069542] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3752.069549] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3752.069554] ? watchdog+0x40/0x40 [ 3752.069559] ? hrtimer_interrupt+0x128/0x2c0 [ 3752.069562] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3752.069571] ? apic_timer_interrupt+0xf/0x20 [ 3752.069576] [ 3752.069579] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3752.069584] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3752.069589] do_raw_spin_lock+0x177/0x190 [ 3752.069594] _raw_spin_lock+0x21/0x30 [ 3752.069600] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3752.069907] ? do_raw_spin_unlock+0x75/0x190 [ 3752.069913] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3752.070128] ? do_wait_intr+0xf0/0xf0 [ 3752.070136] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3752.071400] mdc_reint+0x2d/0x110 [mdc] [ 3752.071438] mdc_unlink+0x658/0x870 [mdc] [ 3752.071465] lmv_unlink+0x34d/0xbd0 [lmv] [ 3752.071493] ll_unlink+0x24d/0x7f0 [lustre] [ 3752.099067] mdc_commit_open+0x91/0xb0 [mdc] [ 3752.099115] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3752.105741] vfs_unlink+0x202/0x330 [ 3752.105760] do_unlinkat+0x2fa/0x460 [ 3752.105766] __x64_sys_unlink+0x37/0x50 [ 3752.105769] do_syscall_64+0xc1/0x440 [ 3752.105777] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3752.105785] RIP: 0033:0x7fe02c11910b [ 3752.105793] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3752.105796] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3752.105802] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3752.105804] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3752.105806] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3752.105809] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3752.105811] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3752.448945] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3752.454700] after_reply+0xe2c/0x1780 [ptlrpc] [ 3752.458939] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3752.461889] ? __next_timer_interrupt+0x160/0x160 [ 3752.465013] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3752.467233] ? do_wait_intr+0xf0/0xf0 [ 3752.469775] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3752.472249] kthread+0x1d1/0x200 [ 3752.473535] ? set_kthread_struct+0x70/0x70 [ 3752.475008] ret_from_fork+0x1f/0x30 [ 3764.064164] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3764.067888] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3764.109855] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3764.117630] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3764.125365] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3764.132261] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3764.137355] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3764.147317] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3764.151119] RAX: 000000000000788d RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3764.154392] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3764.157554] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3764.160345] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3764.163519] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3764.166385] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3764.171409] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3764.176665] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3764.182292] Call Trace: [ 3764.183137] [ 3764.186879] ? show_regs.cold.9+0x22/0x2f [ 3764.191620] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3764.195026] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3764.199837] ? watchdog+0x40/0x40 [ 3764.203676] ? hrtimer_interrupt+0x128/0x2c0 [ 3764.206913] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3764.212178] ? apic_timer_interrupt+0xf/0x20 [ 3764.213650] [ 3764.223357] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3764.228658] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3764.237844] do_raw_spin_lock+0x177/0x190 [ 3764.243774] _raw_spin_lock+0x21/0x30 [ 3764.247948] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3764.256138] process_one_work+0x2c8/0x700 [ 3764.262165] worker_thread+0x296/0x6e0 [ 3764.266887] ? rescuer_thread+0x570/0x570 [ 3764.269491] kthread+0x1d1/0x200 [ 3764.276832] ? set_kthread_struct+0x70/0x70 [ 3764.279316] ret_from_fork+0x1f/0x30 [ 3780.058326] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3780.060172] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3780.064694] Modules linked in: lustre(O) [ 3780.069613] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs [ 3780.071709] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror [ 3780.098921] ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3780.098971] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3780.098981] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3780.098984] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3780.100018] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3780.100024] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3780.100032] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3780.100036] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3780.100039] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3780.100042] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3780.100044] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3780.100049] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3780.100051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3780.100054] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3780.100074] Call Trace: [ 3780.100082] [ 3780.100090] ? show_regs.cold.9+0x22/0x2f [ 3780.100102] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3780.100109] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3780.100114] ? watchdog+0x40/0x40 [ 3780.100119] ? hrtimer_interrupt+0x128/0x2c0 [ 3780.100123] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3780.100133] ? apic_timer_interrupt+0xf/0x20 [ 3780.100138] [ 3780.100142] ? native_safe_halt+0x2e/0x50 [ 3780.100146] kvm_wait+0xcd/0xe0 [ 3780.100157] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3780.100166] do_raw_spin_lock+0x177/0x190 [ 3780.100170] _raw_spin_lock+0x21/0x30 [ 3780.100347] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3780.143254] dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3780.155443] obd_mod_free+0x46/0x140 [ptlrpc] [ 3780.162468] [ 3780.162480] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3780.162490] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3780.162493] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3780.162507] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3780.162511] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 3780.162516] RAX: 000000000000110e RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3780.162518] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3780.162520] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3780.167671] mdc_commit_open+0x91/0xb0 [mdc] [ 3780.167735] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3780.168553] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3780.171499] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3780.171508] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3780.171512] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3780.171515] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3780.171517] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3780.171617] Call Trace: [ 3780.171625] [ 3780.171631] ? show_regs.cold.9+0x22/0x2f [ 3780.171643] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3780.171650] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3780.171655] ? watchdog+0x40/0x40 [ 3780.171659] ? hrtimer_interrupt+0x128/0x2c0 [ 3780.171662] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3780.171672] ? apic_timer_interrupt+0xf/0x20 [ 3780.171676] [ 3780.171680] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3780.171687] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3780.171692] do_raw_spin_lock+0x177/0x190 [ 3780.171697] _raw_spin_lock+0x21/0x30 [ 3780.171704] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3780.171976] ? do_raw_spin_unlock+0x75/0x190 [ 3780.171982] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3780.172180] ? do_wait_intr+0xf0/0xf0 [ 3780.172187] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3780.188453] after_reply+0xe2c/0x1780 [ptlrpc] [ 3780.188733] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3780.188925] ? __next_timer_interrupt+0x160/0x160 [ 3780.188935] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3780.189141] ? do_wait_intr+0xf0/0xf0 [ 3780.189150] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3780.189349] kthread+0x1d1/0x200 [ 3780.189354] ? set_kthread_struct+0x70/0x70 [ 3780.189358] ret_from_fork+0x1f/0x30 [ 3780.417488] mdc_reint+0x2d/0x110 [mdc] [ 3780.418654] mdc_unlink+0x658/0x870 [mdc] [ 3780.420091] lmv_unlink+0x34d/0xbd0 [lmv] [ 3780.421925] ll_unlink+0x24d/0x7f0 [lustre] [ 3780.423873] vfs_unlink+0x202/0x330 [ 3780.425967] do_unlinkat+0x2fa/0x460 [ 3780.427424] __x64_sys_unlink+0x37/0x50 [ 3780.429015] do_syscall_64+0xc1/0x440 [ 3780.430468] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3780.432638] RIP: 0033:0x7fe02c11910b [ 3780.434128] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3780.440301] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3780.443258] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3780.446025] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3780.448920] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3780.451654] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3780.454562] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3792.068872] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3792.077823] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3792.109932] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3792.114306] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3792.118159] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3792.127087] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3792.132509] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3792.143382] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3792.155382] RAX: 0000000000005786 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3792.163917] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3792.171764] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3792.178197] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3792.183585] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3792.187361] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3792.191845] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3792.195399] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3792.198889] Call Trace: [ 3792.199936] [ 3792.201116] ? show_regs.cold.9+0x22/0x2f [ 3792.203248] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3792.206056] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3792.207677] ? watchdog+0x40/0x40 [ 3792.208897] ? hrtimer_interrupt+0x128/0x2c0 [ 3792.210983] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3792.213303] ? apic_timer_interrupt+0xf/0x20 [ 3792.215342] [ 3792.221509] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3792.225350] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3792.229969] do_raw_spin_lock+0x177/0x190 [ 3792.235545] _raw_spin_lock+0x21/0x30 [ 3792.243703] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3792.252448] process_one_work+0x2c8/0x700 [ 3792.257324] worker_thread+0x296/0x6e0 [ 3792.264239] ? rescuer_thread+0x570/0x570 [ 3792.266211] kthread+0x1d1/0x200 [ 3792.267442] ? set_kthread_struct+0x70/0x70 [ 3792.273141] ret_from_fork+0x1f/0x30 [ 3808.057422] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3808.060479] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3808.061152] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3808.061240] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3808.061249] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3808.061253] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3808.061273] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3808.061277] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3808.061344] RAX: 0000000000005a11 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3808.061347] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3808.061349] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3808.061352] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3808.061354] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3808.061358] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3808.061362] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3808.061364] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3808.061374] Call Trace: [ 3808.061381] [ 3808.061388] ? show_regs.cold.9+0x22/0x2f [ 3808.061402] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3808.061409] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3808.061414] ? watchdog+0x40/0x40 [ 3808.061419] ? hrtimer_interrupt+0x128/0x2c0 [ 3808.061424] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3808.061439] ? apic_timer_interrupt+0xf/0x20 [ 3808.061444] [ 3808.061447] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3808.061453] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3808.061457] do_raw_spin_lock+0x177/0x190 [ 3808.061462] _raw_spin_lock+0x21/0x30 [ 3808.061469] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3808.066077] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3808.066182] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3808.066191] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3808.066197] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3808.066216] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3808.066220] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3808.066228] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3808.066231] RDX: 0000000000004a9f RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3808.066233] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3808.066239] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3808.066241] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3808.066244] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3808.066248] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3808.066250] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3808.066258] Call Trace: [ 3808.066266] [ 3808.066272] ? show_regs.cold.9+0x22/0x2f [ 3808.066285] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3808.066291] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3808.066299] ? watchdog+0x40/0x40 [ 3808.066306] ? hrtimer_interrupt+0x128/0x2c0 [ 3808.066316] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3808.066324] ? apic_timer_interrupt+0xf/0x20 [ 3808.066329] [ 3808.066335] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3808.066340] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3808.066345] do_raw_spin_lock+0x177/0x190 [ 3808.066349] _raw_spin_lock+0x21/0x30 [ 3808.066356] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3808.066680] obd_mod_free+0x46/0x140 [ptlrpc] [ 3808.066918] mdc_commit_open+0x91/0xb0 [mdc] [ 3808.066961] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3808.097875] ? do_raw_spin_unlock+0x75/0x190 [ 3808.104448] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3808.104836] after_reply+0xe2c/0x1780 [ptlrpc] [ 3808.105052] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3808.105256] ? __next_timer_interrupt+0x160/0x160 [ 3808.105267] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3808.110503] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3808.110997] ? do_wait_intr+0xf0/0xf0 [ 3808.111022] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3808.112944] ? do_wait_intr+0xf0/0xf0 [ 3808.112962] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3808.113236] kthread+0x1d1/0x200 [ 3808.113241] ? set_kthread_struct+0x70/0x70 [ 3808.113245] ret_from_fork+0x1f/0x30 [ 3808.508286] mdc_reint+0x2d/0x110 [mdc] [ 3808.510171] mdc_unlink+0x658/0x870 [mdc] [ 3808.512256] lmv_unlink+0x34d/0xbd0 [lmv] [ 3808.514478] ll_unlink+0x24d/0x7f0 [lustre] [ 3808.517937] vfs_unlink+0x202/0x330 [ 3808.519507] do_unlinkat+0x2fa/0x460 [ 3808.522143] __x64_sys_unlink+0x37/0x50 [ 3808.527012] do_syscall_64+0xc1/0x440 [ 3808.532205] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3808.544945] RIP: 0033:0x7fe02c11910b [ 3808.549879] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3808.582012] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3808.594096] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3808.604213] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3808.608825] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3808.612868] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3808.618876] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3820.064679] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3820.070164] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3820.113171] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3820.122192] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3820.135529] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3820.145521] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3820.151086] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3820.184582] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3820.197293] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3820.211436] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3820.214026] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3820.232700] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3820.239489] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3820.248921] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3820.252802] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3820.255175] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3820.257881] Call Trace: [ 3820.258987] [ 3820.259676] ? show_regs.cold.9+0x22/0x2f [ 3820.261335] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3820.263277] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3820.269215] ? watchdog+0x40/0x40 [ 3820.270417] ? hrtimer_interrupt+0x128/0x2c0 [ 3820.277058] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3820.279243] ? apic_timer_interrupt+0xf/0x20 [ 3820.281011] [ 3820.281845] ? native_safe_halt+0x2e/0x50 [ 3820.283532] kvm_wait+0xcd/0xe0 [ 3820.284756] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3820.286756] do_raw_spin_lock+0x177/0x190 [ 3820.288421] _raw_spin_lock+0x21/0x30 [ 3820.290005] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3820.292347] process_one_work+0x2c8/0x700 [ 3820.293626] worker_thread+0x296/0x6e0 [ 3820.295140] ? rescuer_thread+0x570/0x570 [ 3820.296861] kthread+0x1d1/0x200 [ 3820.298399] ? set_kthread_struct+0x70/0x70 [ 3820.300233] ret_from_fork+0x1f/0x30 [ 3822.133052] rcu: INFO: rcu_sched self-detected stall on CPU [ 3822.134735] rcu: 1-....: (1108348 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=320835 [ 3822.137623] (t=1680041 jiffies g=278549 q=9657) [ 3822.140035] Sending NMI from CPU 1 to CPUs 0: [ 3822.141813] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 [ 3822.144811] NMI backtrace for cpu 1 [ 3822.146343] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3822.150848] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3822.154411] Call Trace: [ 3822.155380] [ 3822.156221] ? dump_stack+0xbb/0x10e [ 3822.157819] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3822.159819] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3822.161872] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3822.163570] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3822.166187] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3822.168352] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3822.170790] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3822.173447] ? update_process_times+0x6d/0xc0 [ 3822.176053] ? tick_sched_handle.isra.18+0x27/0x90 [ 3822.177959] ? tick_sched_timer+0x53/0xb0 [ 3822.179870] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3822.181934] ? tick_sched_handle.isra.18+0x90/0x90 [ 3822.183288] ? hrtimer_interrupt+0x128/0x2c0 [ 3822.184785] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3822.187027] ? apic_timer_interrupt+0xf/0x20 [ 3822.188829] [ 3822.189713] ? native_safe_halt+0x2e/0x50 [ 3822.191548] ? kvm_wait+0xcd/0xe0 [ 3822.192469] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3822.193824] ? do_raw_spin_lock+0x177/0x190 [ 3822.194987] ? _raw_spin_lock+0x21/0x30 [ 3822.196306] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3822.197878] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3822.199417] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3822.200506] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3822.202673] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3822.205145] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3822.207520] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3822.209760] ? __next_timer_interrupt+0x160/0x160 [ 3822.211805] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3822.213913] ? do_wait_intr+0xf0/0xf0 [ 3822.215620] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3822.217964] ? kthread+0x1d1/0x200 [ 3822.219335] ? set_kthread_struct+0x70/0x70 [ 3822.221233] ? ret_from_fork+0x1f/0x30 [ 3848.057680] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3848.060170] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3848.061843] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3848.061959] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3848.061965] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3848.061968] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3848.061989] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3848.061993] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3848.061998] RAX: 0000000000002684 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3848.062001] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3848.062003] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3848.062005] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3848.062008] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3848.062011] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3848.062014] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3848.064819] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3848.064831] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3848.064898] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3848.064903] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3848.064909] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3848.065711] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3848.065723] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3848.065728] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3848.065732] RAX: 000000000000202f RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3848.065735] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3848.065737] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3848.065739] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3848.065741] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3848.065744] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3848.065746] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3848.065748] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3848.065756] Call Trace: [ 3848.065797] [ 3848.065802] ? show_regs.cold.9+0x22/0x2f [ 3848.065810] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3848.065815] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3848.065819] ? watchdog+0x40/0x40 [ 3848.065824] ? hrtimer_interrupt+0x128/0x2c0 [ 3848.065827] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3848.065834] ? apic_timer_interrupt+0xf/0x20 [ 3848.065838] [ 3848.065841] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3848.065845] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3848.065850] do_raw_spin_lock+0x177/0x190 [ 3848.065854] _raw_spin_lock+0x21/0x30 [ 3848.065857] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3848.066075] process_one_work+0x2c8/0x700 [ 3848.066082] worker_thread+0x296/0x6e0 [ 3848.066086] ? rescuer_thread+0x570/0x570 [ 3848.066089] kthread+0x1d1/0x200 [ 3848.066093] ? set_kthread_struct+0x70/0x70 [ 3848.066097] ret_from_fork+0x1f/0x30 [ 3848.070584] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 [ 3848.104353] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3848.104367] Call Trace: [ 3848.104372] [ 3848.104377] ? show_regs.cold.9+0x22/0x2f [ 3848.104390] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3848.104396] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3848.104399] ? watchdog+0x40/0x40 [ 3848.104403] ? hrtimer_interrupt+0x128/0x2c0 [ 3848.104406] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3848.104646] ? apic_timer_interrupt+0xf/0x20 [ 3848.104651] [ 3848.104654] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3848.104660] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3848.104665] do_raw_spin_lock+0x177/0x190 [ 3848.104670] _raw_spin_lock+0x21/0x30 [ 3848.104677] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3848.105076] ? do_raw_spin_unlock+0x75/0x190 [ 3848.105082] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3848.111115] auth_rpcgss [ 3848.115316] ? do_wait_intr+0xf0/0xf0 [ 3848.115334] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3848.118155] nfsv4 [ 3848.126177] mdc_reint+0x2d/0x110 [mdc] [ 3848.129743] dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3848.129783] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3848.129789] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3848.129802] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3848.129816] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3848.129820] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3848.129824] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3848.129826] RDX: 0000000000005193 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3848.129829] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3848.129831] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3848.129833] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3848.129836] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3848.129839] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3848.129841] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3848.129850] Call Trace: [ 3848.129855] [ 3848.129861] ? show_regs.cold.9+0x22/0x2f [ 3848.129871] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3848.129879] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3848.129884] ? watchdog+0x40/0x40 [ 3848.129887] ? hrtimer_interrupt+0x128/0x2c0 [ 3848.129891] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3848.129899] ? apic_timer_interrupt+0xf/0x20 [ 3848.129903] [ 3848.129906] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3848.129911] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3848.129916] do_raw_spin_lock+0x177/0x190 [ 3848.129920] _raw_spin_lock+0x21/0x30 [ 3848.129926] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3848.130182] obd_mod_free+0x46/0x140 [ptlrpc] [ 3848.130511] mdc_commit_open+0x91/0xb0 [mdc] [ 3848.130552] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3848.137075] mdc_unlink+0x658/0x870 [mdc] [ 3848.140962] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3848.147440] lmv_unlink+0x34d/0xbd0 [lmv] [ 3848.151223] after_reply+0xe2c/0x1780 [ptlrpc] [ 3848.157051] ll_unlink+0x24d/0x7f0 [lustre] [ 3848.160771] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3848.164956] vfs_unlink+0x202/0x330 [ 3848.169056] ? __next_timer_interrupt+0x160/0x160 [ 3848.169076] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3848.169465] ? do_wait_intr+0xf0/0xf0 [ 3848.169474] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3848.169845] kthread+0x1d1/0x200 [ 3848.169851] ? set_kthread_struct+0x70/0x70 [ 3848.169854] ret_from_fork+0x1f/0x30 [ 3848.710657] do_unlinkat+0x2fa/0x460 [ 3848.715841] __x64_sys_unlink+0x37/0x50 [ 3848.718300] do_syscall_64+0xc1/0x440 [ 3848.721216] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3848.729794] RIP: 0033:0x7fe02c11910b [ 3848.733203] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3848.759951] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3848.765431] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3848.767639] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3848.771126] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3848.776651] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3848.780488] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3876.057073] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3876.059160] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3876.060023] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3876.064557] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3876.064689] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3876.064779] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3876.064785] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3876.064792] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3876.065091] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3876.065101] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3876.065104] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 3876.065109] RAX: 0000000000007e99 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3876.065112] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3876.065114] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3876.065116] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3876.065119] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3876.065123] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3876.065125] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3876.065128] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3876.065137] Call Trace: [ 3876.065142] [ 3876.065147] ? show_regs.cold.9+0x22/0x2f [ 3876.065156] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3876.065162] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3876.065166] ? watchdog+0x40/0x40 [ 3876.065171] ? hrtimer_interrupt+0x128/0x2c0 [ 3876.065174] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3876.065182] ? apic_timer_interrupt+0xf/0x20 [ 3876.065187] [ 3876.065190] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3876.065194] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3876.065199] do_raw_spin_lock+0x177/0x190 [ 3876.065203] _raw_spin_lock+0x21/0x30 [ 3876.065208] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3876.066978] process_one_work+0x2c8/0x700 [ 3876.066987] worker_thread+0x296/0x6e0 [ 3876.066992] ? rescuer_thread+0x570/0x570 [ 3876.066995] kthread+0x1d1/0x200 [ 3876.067000] ? set_kthread_struct+0x70/0x70 [ 3876.067003] ret_from_fork+0x1f/0x30 [ 3876.100211] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3876.103083] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3876.103159] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3876.103168] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3876.103171] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3876.103184] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3876.103189] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3876.103193] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3876.103196] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3876.103198] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3876.103200] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3876.103203] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3876.103206] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3876.103209] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3876.103211] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3876.103220] Call Trace: [ 3876.103226] [ 3876.103232] ? show_regs.cold.9+0x22/0x2f [ 3876.103242] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3876.103250] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3876.103254] ? watchdog+0x40/0x40 [ 3876.103259] ? hrtimer_interrupt+0x128/0x2c0 [ 3876.103263] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3876.103271] ? apic_timer_interrupt+0xf/0x20 [ 3876.103276] [ 3876.103279] ? native_safe_halt+0x2e/0x50 [ 3876.103284] kvm_wait+0xcd/0xe0 [ 3876.103292] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3876.103299] do_raw_spin_lock+0x177/0x190 [ 3876.103303] _raw_spin_lock+0x21/0x30 [ 3876.103308] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3876.110586] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3876.128755] obd_mod_free+0x46/0x140 [ptlrpc] [ 3876.129049] mdc_commit_open+0x91/0xb0 [mdc] [ 3876.134998] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3876.138243] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3876.138526] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3876.141080] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3876.159932] after_reply+0xe2c/0x1780 [ptlrpc] [ 3876.178166] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 [ 3876.180424] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3876.185303] ORIG_RAX: ffffffffffffff13 [ 3876.186934] ? __next_timer_interrupt+0x160/0x160 [ 3876.186952] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3876.187226] ? do_wait_intr+0xf0/0xf0 [ 3876.187235] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3876.187392] kthread+0x1d1/0x200 [ 3876.187396] ? set_kthread_struct+0x70/0x70 [ 3876.187398] ret_from_fork+0x1f/0x30 [ 3876.669456] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3876.676556] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3876.688608] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3876.693988] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3876.696231] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3876.705551] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3876.713565] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3876.716590] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3876.719174] Call Trace: [ 3876.729794] [ 3876.730508] ? show_regs.cold.9+0x22/0x2f [ 3876.731465] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3876.732580] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3876.738987] ? watchdog+0x40/0x40 [ 3876.746462] ? hrtimer_interrupt+0x128/0x2c0 [ 3876.751014] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3876.756566] ? apic_timer_interrupt+0xf/0x20 [ 3876.767677] [ 3876.773188] ? native_safe_halt+0x2e/0x50 [ 3876.774596] kvm_wait+0xcd/0xe0 [ 3876.784582] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3876.787525] do_raw_spin_lock+0x177/0x190 [ 3876.797390] _raw_spin_lock+0x21/0x30 [ 3876.800422] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3876.805233] ? do_raw_spin_unlock+0x75/0x190 [ 3876.811464] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3876.816344] ? do_wait_intr+0xf0/0xf0 [ 3876.821508] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3876.826081] mdc_reint+0x2d/0x110 [mdc] [ 3876.830005] mdc_unlink+0x658/0x870 [mdc] [ 3876.831525] lmv_unlink+0x34d/0xbd0 [lmv] [ 3876.844515] ll_unlink+0x24d/0x7f0 [lustre] [ 3876.850047] vfs_unlink+0x202/0x330 [ 3876.852004] do_unlinkat+0x2fa/0x460 [ 3876.857557] __x64_sys_unlink+0x37/0x50 [ 3876.860668] do_syscall_64+0xc1/0x440 [ 3876.864636] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3876.868318] RIP: 0033:0x7fe02c11910b [ 3876.873121] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3876.890790] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3876.899477] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3876.903048] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3876.913327] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3876.918551] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3876.928893] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3904.057046] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 3904.059861] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3904.060977] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 3904.064034] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 3904.064048] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3904.064129] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3904.064135] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3904.064142] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3904.064436] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3904.064445] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3904.064448] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3904.064452] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3904.064455] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3904.064457] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3904.064459] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3904.064462] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3904.064465] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3904.064468] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3904.064470] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3904.064478] Call Trace: [ 3904.064483] [ 3904.064488] ? show_regs.cold.9+0x22/0x2f [ 3904.064499] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3904.064506] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3904.064511] ? watchdog+0x40/0x40 [ 3904.064516] ? hrtimer_interrupt+0x128/0x2c0 [ 3904.064520] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3904.064527] ? apic_timer_interrupt+0xf/0x20 [ 3904.064532] [ 3904.064535] ? native_safe_halt+0x2e/0x50 [ 3904.064539] kvm_wait+0xcd/0xe0 [ 3904.064547] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3904.064555] do_raw_spin_lock+0x177/0x190 [ 3904.064559] _raw_spin_lock+0x21/0x30 [ 3904.064570] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3904.064860] process_one_work+0x2c8/0x700 [ 3904.064867] worker_thread+0x296/0x6e0 [ 3904.064872] ? rescuer_thread+0x570/0x570 [ 3904.064876] kthread+0x1d1/0x200 [ 3904.064880] ? set_kthread_struct+0x70/0x70 [ 3904.064884] ret_from_fork+0x1f/0x30 [ 3904.085677] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3904.088317] Modules linked in: [ 3904.092217] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3904.108384] lustre(O) [ 3904.113877] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3904.116552] osp(O) [ 3904.119384] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3904.120850] ofd(O) [ 3904.137396] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 [ 3904.139833] lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3904.139899] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3904.139906] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3904.139908] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3904.140012] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3904.140016] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3904.140021] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3904.140023] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3904.140025] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3904.140027] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3904.140029] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3904.140032] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3904.140034] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3904.140036] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3904.140043] Call Trace: [ 3904.140048] [ 3904.140053] ? show_regs.cold.9+0x22/0x2f [ 3904.140062] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3904.140068] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3904.140071] ? watchdog+0x40/0x40 [ 3904.140075] ? hrtimer_interrupt+0x128/0x2c0 [ 3904.140078] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3904.140085] ? apic_timer_interrupt+0xf/0x20 [ 3904.140089] [ 3904.140091] ? native_safe_halt+0x2e/0x50 [ 3904.140095] kvm_wait+0xcd/0xe0 [ 3904.140102] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3904.140110] do_raw_spin_lock+0x177/0x190 [ 3904.140113] _raw_spin_lock+0x21/0x30 [ 3904.140118] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3904.140397] obd_mod_free+0x46/0x140 [ptlrpc] [ 3904.140592] mdc_commit_open+0x91/0xb0 [mdc] [ 3904.140687] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3904.140879] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3904.141133] after_reply+0xe2c/0x1780 [ptlrpc] [ 3904.147099] ORIG_RAX: ffffffffffffff13 [ 3904.147109] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3904.147111] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3904.147113] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3904.147115] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3904.147118] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3904.147121] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3904.147124] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3904.147126] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3904.147133] Call Trace: [ 3904.147138] [ 3904.147143] ? show_regs.cold.9+0x22/0x2f [ 3904.147155] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3904.147160] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3904.147164] ? watchdog+0x40/0x40 [ 3904.147168] ? hrtimer_interrupt+0x128/0x2c0 [ 3904.147171] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3904.147179] ? apic_timer_interrupt+0xf/0x20 [ 3904.147183] [ 3904.147186] ? native_safe_halt+0x2e/0x50 [ 3904.147189] kvm_wait+0xcd/0xe0 [ 3904.147196] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3904.147204] do_raw_spin_lock+0x177/0x190 [ 3904.147208] _raw_spin_lock+0x21/0x30 [ 3904.147214] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3904.147538] ? do_raw_spin_unlock+0x75/0x190 [ 3904.147543] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3904.147795] ? do_wait_intr+0xf0/0xf0 [ 3904.147803] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3904.148093] mdc_reint+0x2d/0x110 [mdc] [ 3904.148171] mdc_unlink+0x658/0x870 [mdc] [ 3904.148200] lmv_unlink+0x34d/0xbd0 [lmv] [ 3904.148243] ll_unlink+0x24d/0x7f0 [lustre] [ 3904.148358] vfs_unlink+0x202/0x330 [ 3904.148367] do_unlinkat+0x2fa/0x460 [ 3904.148372] __x64_sys_unlink+0x37/0x50 [ 3904.148375] do_syscall_64+0xc1/0x440 [ 3904.148382] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3904.148389] RIP: 0033:0x7fe02c11910b [ 3904.148397] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3904.148401] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3904.148464] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3904.148466] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3904.148468] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3904.148470] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3904.148473] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3904.384301] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3904.386647] ? __next_timer_interrupt+0x160/0x160 [ 3904.388322] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3904.390326] ? do_wait_intr+0xf0/0xf0 [ 3904.391649] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3904.393961] kthread+0x1d1/0x200 [ 3904.395201] ? set_kthread_struct+0x70/0x70 [ 3904.396665] ret_from_fork+0x1f/0x30 [ 3932.057021] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3932.060238] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3932.064017] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3932.064030] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod [ 3932.064093] Modules linked in: [ 3932.064095] sha512_ssse3 [ 3932.064098] lustre(O) [ 3932.064099] sha512_generic [ 3932.064103] osp(O) ofd(O) lod(O) mdt(O) mdd(O) [ 3932.064111] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3932.064115] mgs(O) lquota(O) [ 3932.064122] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3932.064123] lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3932.064172] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3932.064177] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3932.064179] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3932.064189] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3932.064192] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3932.064196] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3932.064199] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 3932.064200] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3932.064203] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3932.064205] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3932.064208] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3932.064211] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3932.064213] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3932.064220] Call Trace: [ 3932.064225] [ 3932.064230] ? show_regs.cold.9+0x22/0x2f [ 3932.064239] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3932.064246] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3932.064250] ? watchdog+0x40/0x40 [ 3932.064254] ? hrtimer_interrupt+0x128/0x2c0 [ 3932.064257] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3932.064264] ? apic_timer_interrupt+0xf/0x20 [ 3932.064268] [ 3932.064270] ? native_safe_halt+0x2e/0x50 [ 3932.064274] kvm_wait+0xcd/0xe0 [ 3932.064281] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3932.064289] do_raw_spin_lock+0x177/0x190 [ 3932.064131] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3932.064293] _raw_spin_lock+0x21/0x30 [ 3932.064301] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3932.065110] [ 3932.065113] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3932.065126] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3932.065130] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3932.065134] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3932.065137] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3932.065139] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3932.065141] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3932.065143] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3932.065145] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3932.065147] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3932.065149] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3932.065156] Call Trace: [ 3932.065159] [ 3932.065162] ? show_regs.cold.9+0x22/0x2f [ 3932.065170] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3932.065174] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3932.065177] ? watchdog+0x40/0x40 [ 3932.065182] ? hrtimer_interrupt+0x128/0x2c0 [ 3932.065185] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3932.065190] ? apic_timer_interrupt+0xf/0x20 [ 3932.065194] [ 3932.065196] ? native_safe_halt+0x2e/0x50 [ 3932.065200] kvm_wait+0xcd/0xe0 [ 3932.065205] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3932.065211] do_raw_spin_lock+0x177/0x190 [ 3932.065215] _raw_spin_lock+0x21/0x30 [ 3932.065218] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3932.065434] process_one_work+0x2c8/0x700 [ 3932.065441] worker_thread+0x296/0x6e0 [ 3932.065446] ? rescuer_thread+0x570/0x570 [ 3932.065450] kthread+0x1d1/0x200 [ 3932.065455] ? set_kthread_struct+0x70/0x70 [ 3932.065458] ret_from_fork+0x1f/0x30 [ 3932.072639] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3932.072723] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3932.072742] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3932.072746] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3932.072761] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3932.072765] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3932.072770] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3932.072772] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3932.072775] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3932.072777] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3932.072779] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3932.072782] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3932.072784] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3932.072786] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3932.072794] Call Trace: [ 3932.072801] [ 3932.072809] ? show_regs.cold.9+0x22/0x2f [ 3932.072819] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3932.072826] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3932.072830] ? watchdog+0x40/0x40 [ 3932.072834] ? hrtimer_interrupt+0x128/0x2c0 [ 3932.072838] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3932.072900] ? apic_timer_interrupt+0xf/0x20 [ 3932.072905] [ 3932.072908] ? native_safe_halt+0x2e/0x50 [ 3932.072912] kvm_wait+0xcd/0xe0 [ 3932.072979] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3932.073039] do_raw_spin_lock+0x177/0x190 [ 3932.073046] _raw_spin_lock+0x21/0x30 [ 3932.073053] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3932.077623] ? do_raw_spin_unlock+0x75/0x190 [ 3932.096094] obd_mod_free+0x46/0x140 [ptlrpc] [ 3932.097249] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3932.098228] mdc_commit_open+0x91/0xb0 [mdc] [ 3932.099031] ? do_wait_intr+0xf0/0xf0 [ 3932.100196] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3932.102128] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3932.106045] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3932.107098] mdc_reint+0x2d/0x110 [mdc] [ 3932.110106] after_reply+0xe2c/0x1780 [ptlrpc] [ 3932.130389] mdc_unlink+0x658/0x870 [mdc] [ 3932.134688] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3932.138250] lmv_unlink+0x34d/0xbd0 [lmv] [ 3932.140306] ? __next_timer_interrupt+0x160/0x160 [ 3932.146513] ll_unlink+0x24d/0x7f0 [lustre] [ 3932.150198] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3932.150498] ? do_wait_intr+0xf0/0xf0 [ 3932.150509] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3932.150875] kthread+0x1d1/0x200 [ 3932.150882] ? set_kthread_struct+0x70/0x70 [ 3932.150886] ret_from_fork+0x1f/0x30 [ 3932.567645] vfs_unlink+0x202/0x330 [ 3932.570590] do_unlinkat+0x2fa/0x460 [ 3932.572665] __x64_sys_unlink+0x37/0x50 [ 3932.574752] do_syscall_64+0xc1/0x440 [ 3932.576561] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3932.579087] RIP: 0033:0x7fe02c11910b [ 3932.580628] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3932.589265] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3932.592710] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3932.595655] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3932.598851] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3932.601849] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3932.604797] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3960.058791] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3960.060049] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3960.063997] Modules linked in: [ 3960.064393] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3960.064400] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3960.064471] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3960.064477] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3960.064482] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3960.065158] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3960.065170] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3960.065175] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3960.065180] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3960.065182] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3960.065184] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3960.065186] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3960.065188] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3960.065638] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3960.065643] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3960.065646] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3960.065653] Call Trace: [ 3960.065659] [ 3960.065664] ? show_regs.cold.9+0x22/0x2f [ 3960.065675] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3960.065680] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3960.065685] ? watchdog+0x40/0x40 [ 3960.065689] ? hrtimer_interrupt+0x128/0x2c0 [ 3960.065693] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3960.065699] ? apic_timer_interrupt+0xf/0x20 [ 3960.065703] [ 3960.065706] ? native_safe_halt+0x2e/0x50 [ 3960.065709] kvm_wait+0xcd/0xe0 [ 3960.065717] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3960.065725] do_raw_spin_lock+0x177/0x190 [ 3960.065729] _raw_spin_lock+0x21/0x30 [ 3960.065734] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3960.065934] process_one_work+0x2c8/0x700 [ 3960.065939] worker_thread+0x296/0x6e0 [ 3960.065942] ? rescuer_thread+0x570/0x570 [ 3960.065944] kthread+0x1d1/0x200 [ 3960.065948] ? set_kthread_struct+0x70/0x70 [ 3960.065950] ret_from_fork+0x1f/0x30 [ 3960.068249] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3960.068329] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3960.068336] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3960.068338] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3960.068356] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3960.068359] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3960.068364] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3960.068366] RDX: 0000000000002d9c RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3960.068368] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3960.068370] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3960.068373] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3960.068375] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3960.068378] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3960.068381] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3960.068387] Call Trace: [ 3960.068392] [ 3960.068397] ? show_regs.cold.9+0x22/0x2f [ 3960.068407] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3960.068423] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3960.068427] ? watchdog+0x40/0x40 [ 3960.068431] ? hrtimer_interrupt+0x128/0x2c0 [ 3960.068441] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3960.068449] ? apic_timer_interrupt+0xf/0x20 [ 3960.068455] [ 3960.068458] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3960.068463] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3960.068468] do_raw_spin_lock+0x177/0x190 [ 3960.068472] _raw_spin_lock+0x21/0x30 [ 3960.068481] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3960.068783] obd_mod_free+0x46/0x140 [ptlrpc] [ 3960.069141] mdc_commit_open+0x91/0xb0 [mdc] [ 3960.069180] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3960.074400] lustre(O) osp(O) [ 3960.079602] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3960.080156] after_reply+0xe2c/0x1780 [ptlrpc] [ 3960.080484] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3960.080917] ? __next_timer_interrupt+0x160/0x160 [ 3960.080930] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3960.081294] ? do_wait_intr+0xf0/0xf0 [ 3960.081304] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3960.081752] kthread+0x1d1/0x200 [ 3960.081759] ? set_kthread_struct+0x70/0x70 [ 3960.081763] ret_from_fork+0x1f/0x30 [ 3960.341074] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3960.361897] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3960.366229] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3960.370453] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3960.373580] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3960.383863] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3960.387994] RAX: 0000000000001694 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3960.391577] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3960.394897] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3960.398678] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3960.401560] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3960.403928] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3960.407293] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3960.410059] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3960.413314] Call Trace: [ 3960.414437] [ 3960.415431] ? show_regs.cold.9+0x22/0x2f [ 3960.417331] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3960.419449] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3960.422225] ? watchdog+0x40/0x40 [ 3960.424132] ? hrtimer_interrupt+0x128/0x2c0 [ 3960.426083] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3960.428310] ? apic_timer_interrupt+0xf/0x20 [ 3960.431184] [ 3960.432850] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3960.436807] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3960.439980] do_raw_spin_lock+0x177/0x190 [ 3960.442906] _raw_spin_lock+0x21/0x30 [ 3960.445149] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3960.446669] ? do_raw_spin_unlock+0x75/0x190 [ 3960.447938] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3960.449778] ? do_wait_intr+0xf0/0xf0 [ 3960.451050] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3960.453222] mdc_reint+0x2d/0x110 [mdc] [ 3960.454688] mdc_unlink+0x658/0x870 [mdc] [ 3960.456218] lmv_unlink+0x34d/0xbd0 [lmv] [ 3960.457862] ll_unlink+0x24d/0x7f0 [lustre] [ 3960.459468] vfs_unlink+0x202/0x330 [ 3960.461000] do_unlinkat+0x2fa/0x460 [ 3960.462380] __x64_sys_unlink+0x37/0x50 [ 3960.463888] do_syscall_64+0xc1/0x440 [ 3960.476315] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3960.480388] RIP: 0033:0x7fe02c11910b [ 3960.482323] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3960.490270] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3960.492925] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3960.494953] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3960.497491] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3960.500141] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3960.502785] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3988.059132] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 3988.063927] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 3988.071407] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 3988.071421] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3988.071494] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3988.071500] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3988.071506] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3988.071874] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3988.071885] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3988.071888] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3988.071893] RAX: 000000000000691d RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3988.071895] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 3988.071897] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3988.071899] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 3988.071900] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 3988.071904] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 3988.071906] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3988.071907] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 3988.071915] Call Trace: [ 3988.071921] [ 3988.071926] ? show_regs.cold.9+0x22/0x2f [ 3988.071934] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3988.071946] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3988.071951] ? watchdog+0x40/0x40 [ 3988.071954] ? hrtimer_interrupt+0x128/0x2c0 [ 3988.071958] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3988.071967] ? apic_timer_interrupt+0xf/0x20 [ 3988.071971] [ 3988.071974] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3988.071979] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3988.071983] do_raw_spin_lock+0x177/0x190 [ 3988.071988] _raw_spin_lock+0x21/0x30 [ 3988.071992] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3988.072253] process_one_work+0x2c8/0x700 [ 3988.072260] worker_thread+0x296/0x6e0 [ 3988.072375] ? rescuer_thread+0x570/0x570 [ 3988.072382] kthread+0x1d1/0x200 [ 3988.072390] ? set_kthread_struct+0x70/0x70 [ 3988.072394] ret_from_fork+0x1f/0x30 [ 3988.076290] Modules linked in: [ 3988.084286] Modules linked in: lustre(O) [ 3988.087806] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3988.087914] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3988.087922] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3988.087925] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3988.087945] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3988.087948] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3988.087953] RAX: 000000000000590c RBX: ffff99374626eae0 RCX: 0000000000000001 [ 3988.087956] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 3988.087958] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 3988.087960] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 3988.087962] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 3988.087965] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 3988.087968] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3988.087970] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 3988.087978] Call Trace: [ 3988.087983] [ 3988.087988] ? show_regs.cold.9+0x22/0x2f [ 3988.088009] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3988.088015] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3988.088019] ? watchdog+0x40/0x40 [ 3988.088023] ? hrtimer_interrupt+0x128/0x2c0 [ 3988.088031] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3988.088038] ? apic_timer_interrupt+0xf/0x20 [ 3988.088043] [ 3988.088046] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3988.088050] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3988.088055] do_raw_spin_lock+0x177/0x190 [ 3988.088060] _raw_spin_lock+0x21/0x30 [ 3988.088068] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3988.088358] ? do_raw_spin_unlock+0x75/0x190 [ 3988.088364] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3988.088943] ? do_wait_intr+0xf0/0xf0 [ 3988.088952] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3988.089172] mdc_reint+0x2d/0x110 [mdc] [ 3988.089211] mdc_unlink+0x658/0x870 [mdc] [ 3988.089238] lmv_unlink+0x34d/0xbd0 [lmv] [ 3988.089275] ll_unlink+0x24d/0x7f0 [lustre] [ 3988.089409] vfs_unlink+0x202/0x330 [ 3988.089422] do_unlinkat+0x2fa/0x460 [ 3988.089426] __x64_sys_unlink+0x37/0x50 [ 3988.089430] do_syscall_64+0xc1/0x440 [ 3988.089436] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3988.089443] RIP: 0033:0x7fe02c11910b [ 3988.089450] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3988.089454] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3988.089459] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 3988.089461] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 3988.089463] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 3988.089465] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 3988.089467] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3988.471220] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3988.506049] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3988.515341] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3988.518953] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3988.522617] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3988.531450] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3988.536482] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 3988.539042] RDX: 0000000000007cea RSI: 0000000000000003 RDI: ffff99374626eae0 [ 3988.544577] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 3988.548731] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 3988.559035] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3988.561162] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 3988.565083] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3988.570630] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 3988.573799] Call Trace: [ 3988.574918] [ 3988.577960] ? show_regs.cold.9+0x22/0x2f [ 3988.582671] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3988.586504] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3988.588527] ? watchdog+0x40/0x40 [ 3988.591873] ? hrtimer_interrupt+0x128/0x2c0 [ 3988.595432] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3988.596896] ? apic_timer_interrupt+0xf/0x20 [ 3988.598420] [ 3988.599084] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3988.601407] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3988.604831] do_raw_spin_lock+0x177/0x190 [ 3988.607299] _raw_spin_lock+0x21/0x30 [ 3988.609290] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3988.612280] obd_mod_free+0x46/0x140 [ptlrpc] [ 3988.615093] mdc_commit_open+0x91/0xb0 [mdc] [ 3988.618680] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3988.622083] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3988.625060] after_reply+0xe2c/0x1780 [ptlrpc] [ 3988.629025] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3988.632377] ? __next_timer_interrupt+0x160/0x160 [ 3988.635672] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3988.638848] ? do_wait_intr+0xf0/0xf0 [ 3988.640704] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3988.643892] kthread+0x1d1/0x200 [ 3988.647571] ? set_kthread_struct+0x70/0x70 [ 3988.650691] ret_from_fork+0x1f/0x30 [ 4002.136016] rcu: INFO: rcu_sched self-detected stall on CPU [ 4002.138662] rcu: 1-....: (1236131 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=356529 [ 4002.143297] (t=1860044 jiffies g=278549 q=10332) [ 4002.144982] Sending NMI from CPU 1 to CPUs 0: [ 4002.146667] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 [ 4002.147634] NMI backtrace for cpu 1 [ 4002.153780] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4002.159100] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4002.163471] Call Trace: [ 4002.165453] [ 4002.165956] ? dump_stack+0xbb/0x10e [ 4002.167315] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4002.168787] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4002.170588] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4002.172273] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4002.173622] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4002.175597] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4002.177426] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4002.179291] ? update_process_times+0x6d/0xc0 [ 4002.180965] ? tick_sched_handle.isra.18+0x27/0x90 [ 4002.182884] ? tick_sched_timer+0x53/0xb0 [ 4002.184758] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4002.187063] ? tick_sched_handle.isra.18+0x90/0x90 [ 4002.189473] ? hrtimer_interrupt+0x128/0x2c0 [ 4002.191660] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4002.194469] ? apic_timer_interrupt+0xf/0x20 [ 4002.196450] [ 4002.197552] ? native_safe_halt+0x2e/0x50 [ 4002.199415] ? kvm_wait+0xcd/0xe0 [ 4002.200662] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4002.203379] ? do_raw_spin_lock+0x177/0x190 [ 4002.205199] ? _raw_spin_lock+0x21/0x30 [ 4002.206880] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4002.209508] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4002.211781] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4002.213858] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4002.216852] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4002.219828] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4002.222542] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4002.225559] ? __next_timer_interrupt+0x160/0x160 [ 4002.227947] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4002.230489] ? do_wait_intr+0xf0/0xf0 [ 4002.232444] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4002.235304] ? kthread+0x1d1/0x200 [ 4002.236534] ? set_kthread_struct+0x70/0x70 [ 4002.238197] ? ret_from_fork+0x1f/0x30 [ 4016.064229] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4016.070880] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4016.106285] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4016.112029] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4016.115418] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4016.118229] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4016.120194] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4016.147871] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4016.152549] RAX: 0000000000005395 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4016.166295] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4016.171328] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4016.173866] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4016.176868] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4016.180193] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4016.184103] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4016.187347] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4016.192239] Call Trace: [ 4016.193702] [ 4016.194306] ? show_regs.cold.9+0x22/0x2f [ 4016.196048] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4016.200480] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4016.202627] ? watchdog+0x40/0x40 [ 4016.204167] ? hrtimer_interrupt+0x128/0x2c0 [ 4016.205495] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4016.210803] ? apic_timer_interrupt+0xf/0x20 [ 4016.213178] [ 4016.215189] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4016.218468] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4016.220604] do_raw_spin_lock+0x177/0x190 [ 4016.223272] _raw_spin_lock+0x21/0x30 [ 4016.232538] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4016.236913] process_one_work+0x2c8/0x700 [ 4016.238106] worker_thread+0x296/0x6e0 [ 4016.241338] ? rescuer_thread+0x570/0x570 [ 4016.243236] kthread+0x1d1/0x200 [ 4016.244361] ? set_kthread_struct+0x70/0x70 [ 4016.246313] ret_from_fork+0x1f/0x30 [ 4028.058246] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4028.063264] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4028.065384] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4028.105182] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4028.105197] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4028.105200] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4028.105265] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4028.105269] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4028.105275] RAX: 0000000000002115 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4028.105278] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4028.105281] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4028.105282] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4028.105285] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4028.105289] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4028.105292] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4028.105294] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4028.105303] Call Trace: [ 4028.105320] [ 4028.105327] ? show_regs.cold.9+0x22/0x2f [ 4028.105337] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4028.105344] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4028.105348] ? watchdog+0x40/0x40 [ 4028.105352] ? hrtimer_interrupt+0x128/0x2c0 [ 4028.105356] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4028.105365] ? apic_timer_interrupt+0xf/0x20 [ 4028.105369] [ 4028.105372] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4028.105377] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4028.105383] do_raw_spin_lock+0x177/0x190 [ 4028.105387] _raw_spin_lock+0x21/0x30 [ 4028.105394] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4028.105995] ? do_raw_spin_unlock+0x75/0x190 [ 4028.106002] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4028.106221] ? do_wait_intr+0xf0/0xf0 [ 4028.106229] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4028.106437] mdc_reint+0x2d/0x110 [mdc] [ 4028.106663] mdc_unlink+0x658/0x870 [mdc] [ 4028.106760] lmv_unlink+0x34d/0xbd0 [lmv] [ 4028.106791] ll_unlink+0x24d/0x7f0 [lustre] [ 4028.106907] vfs_unlink+0x202/0x330 [ 4028.106914] do_unlinkat+0x2fa/0x460 [ 4028.106919] __x64_sys_unlink+0x37/0x50 [ 4028.106922] do_syscall_64+0xc1/0x440 [ 4028.106927] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4028.106934] RIP: 0033:0x7fe02c11910b [ 4028.106941] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4028.106944] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4028.106948] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4028.106951] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4028.106952] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4028.106954] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4028.106956] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4028.387184] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4028.417793] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4028.431961] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4028.436210] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4028.440891] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4028.455457] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4028.462895] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4028.468040] RDX: 0000000000006cf1 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4028.472209] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4028.475479] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4028.478132] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4028.481348] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4028.484580] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4028.487016] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4028.490788] Call Trace: [ 4028.491818] [ 4028.492798] ? show_regs.cold.9+0x22/0x2f [ 4028.494727] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4028.496903] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4028.498660] ? watchdog+0x40/0x40 [ 4028.499922] ? hrtimer_interrupt+0x128/0x2c0 [ 4028.501611] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4028.503621] ? apic_timer_interrupt+0xf/0x20 [ 4028.505529] [ 4028.506505] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4028.510180] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4028.514162] do_raw_spin_lock+0x177/0x190 [ 4028.516339] _raw_spin_lock+0x21/0x30 [ 4028.518346] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4028.522029] obd_mod_free+0x46/0x140 [ptlrpc] [ 4028.525794] mdc_commit_open+0x91/0xb0 [mdc] [ 4028.531810] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4028.535235] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4028.541187] after_reply+0xe2c/0x1780 [ptlrpc] [ 4028.545988] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4028.548773] ? __next_timer_interrupt+0x160/0x160 [ 4028.553391] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4028.555587] ? do_wait_intr+0xf0/0xf0 [ 4028.557335] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4028.561915] kthread+0x1d1/0x200 [ 4028.564992] ? set_kthread_struct+0x70/0x70 [ 4028.566241] ret_from_fork+0x1f/0x30 [ 4044.064292] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4044.068658] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4044.111575] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4044.116966] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4044.122912] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4044.127619] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4044.131751] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4044.151696] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4044.161552] RAX: 0000000000007d1c RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4044.170352] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4044.178333] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4044.186692] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4044.194870] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4044.200672] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4044.204902] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4044.212000] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4044.214549] Call Trace: [ 4044.220695] [ 4044.222629] ? show_regs.cold.9+0x22/0x2f [ 4044.224531] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4044.226854] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4044.231511] ? watchdog+0x40/0x40 [ 4044.232901] ? hrtimer_interrupt+0x128/0x2c0 [ 4044.237207] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4044.238973] ? apic_timer_interrupt+0xf/0x20 [ 4044.240456] [ 4044.245858] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4044.250897] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4044.253751] do_raw_spin_lock+0x177/0x190 [ 4044.259589] _raw_spin_lock+0x21/0x30 [ 4044.264532] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4044.268095] process_one_work+0x2c8/0x700 [ 4044.270718] worker_thread+0x296/0x6e0 [ 4044.272576] ? rescuer_thread+0x570/0x570 [ 4044.275046] kthread+0x1d1/0x200 [ 4044.278354] ? set_kthread_struct+0x70/0x70 [ 4044.281644] ret_from_fork+0x1f/0x30 [ 4056.057051] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4056.060677] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4056.061180] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4056.061253] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4056.061259] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4056.061261] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4056.061274] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4056.061278] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4056.061283] RAX: 00000000000004e3 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4056.061285] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4056.061287] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4056.061288] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4056.061513] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4056.061517] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4056.061519] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4056.061522] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4056.061528] Call Trace: [ 4056.061534] [ 4056.061539] ? show_regs.cold.9+0x22/0x2f [ 4056.061547] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4056.061553] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4056.061557] ? watchdog+0x40/0x40 [ 4056.061572] ? hrtimer_interrupt+0x128/0x2c0 [ 4056.061575] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4056.061582] ? apic_timer_interrupt+0xf/0x20 [ 4056.061586] [ 4056.061588] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4056.061592] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4056.061596] do_raw_spin_lock+0x177/0x190 [ 4056.061600] _raw_spin_lock+0x21/0x30 [ 4056.061605] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4056.062023] ? do_raw_spin_unlock+0x75/0x190 [ 4056.062028] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4056.062215] ? do_wait_intr+0xf0/0xf0 [ 4056.062222] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4056.062523] mdc_reint+0x2d/0x110 [mdc] [ 4056.062586] mdc_unlink+0x658/0x870 [mdc] [ 4056.062611] lmv_unlink+0x34d/0xbd0 [lmv] [ 4056.062641] ll_unlink+0x24d/0x7f0 [lustre] [ 4056.062794] vfs_unlink+0x202/0x330 [ 4056.062801] do_unlinkat+0x2fa/0x460 [ 4056.062806] __x64_sys_unlink+0x37/0x50 [ 4056.062808] do_syscall_64+0xc1/0x440 [ 4056.062814] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4056.062820] RIP: 0033:0x7fe02c11910b [ 4056.062826] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4056.062829] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4056.062833] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4056.062835] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4056.062837] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4056.062839] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4056.062841] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4056.229003] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4056.246090] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4056.250631] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4056.253681] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4056.256231] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4056.262739] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4056.265172] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4056.267566] RDX: 0000000000000a0d RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4056.269659] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4056.271651] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4056.273713] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4056.275495] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4056.278333] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4056.280432] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4056.282883] Call Trace: [ 4056.283640] [ 4056.284319] ? show_regs.cold.9+0x22/0x2f [ 4056.285410] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4056.287433] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4056.288890] ? watchdog+0x40/0x40 [ 4056.290475] ? hrtimer_interrupt+0x128/0x2c0 [ 4056.291485] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4056.292869] ? apic_timer_interrupt+0xf/0x20 [ 4056.295872] [ 4056.296692] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4056.299906] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4056.302005] do_raw_spin_lock+0x177/0x190 [ 4056.303206] _raw_spin_lock+0x21/0x30 [ 4056.304845] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4056.306342] obd_mod_free+0x46/0x140 [ptlrpc] [ 4056.309930] mdc_commit_open+0x91/0xb0 [mdc] [ 4056.311253] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4056.313145] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4056.314873] after_reply+0xe2c/0x1780 [ptlrpc] [ 4056.316610] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4056.318168] ? __next_timer_interrupt+0x160/0x160 [ 4056.321066] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4056.323660] ? do_wait_intr+0xf0/0xf0 [ 4056.325015] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4056.327303] kthread+0x1d1/0x200 [ 4056.328422] ? set_kthread_struct+0x70/0x70 [ 4056.330371] ret_from_fork+0x1f/0x30 [ 4072.064561] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4072.066997] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4072.100538] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4072.105162] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4072.114185] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4072.119292] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4072.123485] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4072.141202] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4072.144936] RAX: 0000000000000b87 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4072.149007] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4072.153051] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4072.154892] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4072.157830] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4072.161221] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4072.164914] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4072.167409] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4072.170492] Call Trace: [ 4072.171551] [ 4072.172464] ? show_regs.cold.9+0x22/0x2f [ 4072.173844] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4072.175551] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4072.177449] ? watchdog+0x40/0x40 [ 4072.178889] ? hrtimer_interrupt+0x128/0x2c0 [ 4072.180532] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4072.182576] ? apic_timer_interrupt+0xf/0x20 [ 4072.184508] [ 4072.185518] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4072.187558] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4072.190813] do_raw_spin_lock+0x177/0x190 [ 4072.192200] _raw_spin_lock+0x21/0x30 [ 4072.195629] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4072.199099] process_one_work+0x2c8/0x700 [ 4072.201368] worker_thread+0x296/0x6e0 [ 4072.205359] ? rescuer_thread+0x570/0x570 [ 4072.206885] kthread+0x1d1/0x200 [ 4072.208396] ? set_kthread_struct+0x70/0x70 [ 4072.209903] ret_from_fork+0x1f/0x30 [ 4084.057023] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4084.060019] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4084.061514] Modules linked in: lustre(O) osp(O) [ 4084.063815] Modules linked in: lustre(O) osp(O) ofd(O) [ 4084.068569] ofd(O) lod(O) [ 4084.070152] lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4084.070220] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4084.070227] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4084.070230] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4084.070243] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4084.070247] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4084.070257] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4084.071602] mdt(O) [ 4084.084334] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4084.084344] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4084.084347] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4084.088749] mdd(O) [ 4084.091272] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4084.092877] mgs(O) [ 4084.099166] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4084.099176] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4084.099178] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4084.099187] Call Trace: [ 4084.099193] [ 4084.099198] ? show_regs.cold.9+0x22/0x2f [ 4084.099212] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4084.099218] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4084.099222] ? watchdog+0x40/0x40 [ 4084.099227] ? hrtimer_interrupt+0x128/0x2c0 [ 4084.099231] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4084.099240] ? apic_timer_interrupt+0xf/0x20 [ 4084.099244] [ 4084.099247] ? native_safe_halt+0x2e/0x50 [ 4084.099252] kvm_wait+0xcd/0xe0 [ 4084.099260] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4084.099267] do_raw_spin_lock+0x177/0x190 [ 4084.099272] _raw_spin_lock+0x21/0x30 [ 4084.099278] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4084.099599] obd_mod_free+0x46/0x140 [ptlrpc] [ 4084.099819] mdc_commit_open+0x91/0xb0 [mdc] [ 4084.099855] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4084.100047] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4084.100255] after_reply+0xe2c/0x1780 [ptlrpc] [ 4084.100619] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4084.109663] lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4084.109728] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4084.109736] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4084.109738] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4084.109752] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4084.109757] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4084.109762] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4084.109764] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4084.109767] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4084.109769] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4084.109771] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4084.109775] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4084.109778] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4084.109781] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4084.109789] Call Trace: [ 4084.109794] [ 4084.109801] ? show_regs.cold.9+0x22/0x2f [ 4084.109812] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4084.109818] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4084.109822] ? watchdog+0x40/0x40 [ 4084.109826] ? hrtimer_interrupt+0x128/0x2c0 [ 4084.109831] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4084.109837] ? apic_timer_interrupt+0xf/0x20 [ 4084.109842] [ 4084.109845] ? native_safe_halt+0x2e/0x50 [ 4084.109855] kvm_wait+0xcd/0xe0 [ 4084.109864] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4084.109871] do_raw_spin_lock+0x177/0x190 [ 4084.109876] _raw_spin_lock+0x21/0x30 [ 4084.109881] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4084.110170] ? do_raw_spin_unlock+0x75/0x190 [ 4084.110176] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4084.110405] ? do_wait_intr+0xf0/0xf0 [ 4084.110417] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4084.110682] mdc_reint+0x2d/0x110 [mdc] [ 4084.110719] mdc_unlink+0x658/0x870 [mdc] [ 4084.110747] lmv_unlink+0x34d/0xbd0 [lmv] [ 4084.110782] ll_unlink+0x24d/0x7f0 [lustre] [ 4084.110896] vfs_unlink+0x202/0x330 [ 4084.110903] do_unlinkat+0x2fa/0x460 [ 4084.110909] __x64_sys_unlink+0x37/0x50 [ 4084.110912] do_syscall_64+0xc1/0x440 [ 4084.110919] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4084.110926] RIP: 0033:0x7fe02c11910b [ 4084.110932] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4084.110936] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4084.110940] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4084.110942] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4084.110945] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4084.110947] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4084.110949] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4084.282167] ? __next_timer_interrupt+0x160/0x160 [ 4084.283975] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4084.285623] ? do_wait_intr+0xf0/0xf0 [ 4084.286903] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4084.288838] kthread+0x1d1/0x200 [ 4084.290120] ? set_kthread_struct+0x70/0x70 [ 4084.291431] ret_from_fork+0x1f/0x30 [ 4100.065761] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4100.072653] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4100.112749] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4100.122127] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4100.125445] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4100.127804] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4100.133987] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4100.146798] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4100.151332] RAX: 0000000000007f89 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4100.158184] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4100.163661] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4100.172131] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4100.177084] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4100.183463] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4100.187418] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4100.190156] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4100.195055] Call Trace: [ 4100.196099] [ 4100.198445] ? show_regs.cold.9+0x22/0x2f [ 4100.201884] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4100.205105] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4100.207129] ? watchdog+0x40/0x40 [ 4100.208301] ? hrtimer_interrupt+0x128/0x2c0 [ 4100.210490] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4100.214980] ? apic_timer_interrupt+0xf/0x20 [ 4100.219989] [ 4100.220567] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4100.223450] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4100.230824] do_raw_spin_lock+0x177/0x190 [ 4100.232956] _raw_spin_lock+0x21/0x30 [ 4100.235578] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4100.239448] process_one_work+0x2c8/0x700 [ 4100.244116] worker_thread+0x296/0x6e0 [ 4100.249268] ? rescuer_thread+0x570/0x570 [ 4100.250666] kthread+0x1d1/0x200 [ 4100.253621] ? set_kthread_struct+0x70/0x70 [ 4100.258320] ret_from_fork+0x1f/0x30 [ 4112.057049] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4112.060556] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4112.062593] Modules linked in: [ 4112.067704] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4112.067778] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4112.067785] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4112.067788] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4112.068991] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4112.069058] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4112.069065] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4112.069067] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4112.069078] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4112.069083] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4112.069087] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4112.069090] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4112.069092] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4112.069094] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4112.069097] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4112.069100] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4112.069103] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4112.069105] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4112.069114] Call Trace: [ 4112.069120] [ 4112.069126] ? show_regs.cold.9+0x22/0x2f [ 4112.069136] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4112.069142] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4112.069147] ? watchdog+0x40/0x40 [ 4112.069152] ? hrtimer_interrupt+0x128/0x2c0 [ 4112.069155] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4112.069163] ? apic_timer_interrupt+0xf/0x20 [ 4112.069168] [ 4112.069171] ? native_safe_halt+0x2e/0x50 [ 4112.069175] kvm_wait+0xcd/0xe0 [ 4112.069183] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4112.069191] do_raw_spin_lock+0x177/0x190 [ 4112.069195] _raw_spin_lock+0x21/0x30 [ 4112.069200] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4112.069502] ? do_raw_spin_unlock+0x75/0x190 [ 4112.069508] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4112.069788] ? do_wait_intr+0xf0/0xf0 [ 4112.069796] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4112.070000] mdc_reint+0x2d/0x110 [mdc] [ 4112.070041] mdc_unlink+0x658/0x870 [mdc] [ 4112.070069] lmv_unlink+0x34d/0xbd0 [lmv] [ 4112.070103] ll_unlink+0x24d/0x7f0 [lustre] [ 4112.070229] vfs_unlink+0x202/0x330 [ 4112.070238] do_unlinkat+0x2fa/0x460 [ 4112.070243] __x64_sys_unlink+0x37/0x50 [ 4112.070246] do_syscall_64+0xc1/0x440 [ 4112.070253] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4112.070260] RIP: 0033:0x7fe02c11910b [ 4112.070268] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4112.070273] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4112.070278] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4112.070280] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4112.070282] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4112.070284] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4112.070286] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4112.274844] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4112.282117] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4112.286072] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4112.289464] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4112.292522] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4112.295456] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4112.298427] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4112.301646] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4112.305759] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4112.308294] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4112.311466] Call Trace: [ 4112.312546] [ 4112.313485] ? show_regs.cold.9+0x22/0x2f [ 4112.315375] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4112.317537] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4112.319978] ? watchdog+0x40/0x40 [ 4112.321469] ? hrtimer_interrupt+0x128/0x2c0 [ 4112.323846] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4112.326204] ? apic_timer_interrupt+0xf/0x20 [ 4112.328179] [ 4112.329278] ? native_safe_halt+0x2e/0x50 [ 4112.331466] kvm_wait+0xcd/0xe0 [ 4112.333432] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4112.336253] do_raw_spin_lock+0x177/0x190 [ 4112.338329] _raw_spin_lock+0x21/0x30 [ 4112.340202] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4112.342935] obd_mod_free+0x46/0x140 [ptlrpc] [ 4112.345521] mdc_commit_open+0x91/0xb0 [mdc] [ 4112.347482] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4112.350363] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4112.353110] after_reply+0xe2c/0x1780 [ptlrpc] [ 4112.356257] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4112.359641] ? __next_timer_interrupt+0x160/0x160 [ 4112.361643] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4112.364094] ? do_wait_intr+0xf0/0xf0 [ 4112.366249] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4112.369394] kthread+0x1d1/0x200 [ 4112.371027] ? set_kthread_struct+0x70/0x70 [ 4112.372914] ret_from_fork+0x1f/0x30 [ 4128.065936] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4128.071568] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4128.111426] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4128.122536] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4128.127352] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4128.131505] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4128.142403] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4128.162502] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4128.167322] RAX: 00000000000025ec RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4128.174636] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4128.182486] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4128.186350] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4128.190230] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4128.194053] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4128.197494] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4128.201533] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4128.210581] Call Trace: [ 4128.211385] [ 4128.212043] ? show_regs.cold.9+0x22/0x2f [ 4128.215504] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4128.217363] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4128.219668] ? watchdog+0x40/0x40 [ 4128.225499] ? hrtimer_interrupt+0x128/0x2c0 [ 4128.229754] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4128.231629] ? apic_timer_interrupt+0xf/0x20 [ 4128.233914] [ 4128.234694] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4128.236629] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4128.238918] do_raw_spin_lock+0x177/0x190 [ 4128.241020] _raw_spin_lock+0x21/0x30 [ 4128.242469] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4128.246012] process_one_work+0x2c8/0x700 [ 4128.252583] worker_thread+0x296/0x6e0 [ 4128.258155] ? rescuer_thread+0x570/0x570 [ 4128.260975] kthread+0x1d1/0x200 [ 4128.262120] ? set_kthread_struct+0x70/0x70 [ 4128.265164] ret_from_fork+0x1f/0x30 [ 4140.058016] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4140.062363] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4140.067103] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4140.067186] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4140.067193] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4140.067196] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4140.067211] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4140.067215] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4140.067220] RAX: 000000000000395c RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4140.067222] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4140.067224] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4140.067227] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4140.067229] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4140.067233] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4140.067236] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4140.067238] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4140.067246] Call Trace: [ 4140.067251] [ 4140.067256] ? show_regs.cold.9+0x22/0x2f [ 4140.067264] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4140.067270] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4140.067274] ? watchdog+0x40/0x40 [ 4140.067278] ? hrtimer_interrupt+0x128/0x2c0 [ 4140.067282] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4140.067289] ? apic_timer_interrupt+0xf/0x20 [ 4140.067293] [ 4140.067296] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4140.067300] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4140.067304] do_raw_spin_lock+0x177/0x190 [ 4140.067309] _raw_spin_lock+0x21/0x30 [ 4140.067314] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4140.067937] ? do_raw_spin_unlock+0x75/0x190 [ 4140.067944] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4140.072319] Modules linked in: lustre(O) [ 4140.118995] ? do_wait_intr+0xf0/0xf0 [ 4140.119016] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4140.119537] mdc_reint+0x2d/0x110 [mdc] [ 4140.119578] mdc_unlink+0x658/0x870 [mdc] [ 4140.119605] lmv_unlink+0x34d/0xbd0 [lmv] [ 4140.119849] ll_unlink+0x24d/0x7f0 [lustre] [ 4140.119968] vfs_unlink+0x202/0x330 [ 4140.119977] do_unlinkat+0x2fa/0x460 [ 4140.119982] __x64_sys_unlink+0x37/0x50 [ 4140.119985] do_syscall_64+0xc1/0x440 [ 4140.119991] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4140.120000] RIP: 0033:0x7fe02c11910b [ 4140.120008] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4140.120011] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4140.120017] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4140.120019] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4140.120021] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4140.120023] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4140.120025] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4140.378079] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4140.421186] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4140.429570] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4140.435783] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4140.442073] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4140.458598] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4140.465978] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4140.468668] RDX: 0000000000007d5e RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4140.473176] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4140.478029] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4140.485608] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4140.490962] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4140.494159] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4140.497516] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4140.502894] Call Trace: [ 4140.504864] [ 4140.505423] ? show_regs.cold.9+0x22/0x2f [ 4140.506296] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4140.507314] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4140.511075] ? watchdog+0x40/0x40 [ 4140.514590] ? hrtimer_interrupt+0x128/0x2c0 [ 4140.517223] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4140.520058] ? apic_timer_interrupt+0xf/0x20 [ 4140.523058] [ 4140.524880] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4140.526053] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4140.527683] do_raw_spin_lock+0x177/0x190 [ 4140.529576] _raw_spin_lock+0x21/0x30 [ 4140.531781] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4140.534833] obd_mod_free+0x46/0x140 [ptlrpc] [ 4140.539424] mdc_commit_open+0x91/0xb0 [mdc] [ 4140.546858] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4140.555584] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4140.559355] after_reply+0xe2c/0x1780 [ptlrpc] [ 4140.563135] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4140.567232] ? __next_timer_interrupt+0x160/0x160 [ 4140.569903] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4140.571578] ? do_wait_intr+0xf0/0xf0 [ 4140.572824] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4140.578314] kthread+0x1d1/0x200 [ 4140.579455] ? set_kthread_struct+0x70/0x70 [ 4140.581231] ret_from_fork+0x1f/0x30 [ 4156.064109] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4156.067556] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4156.096118] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4156.103493] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4156.107141] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4156.111177] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4156.114491] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4156.124496] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4156.128772] RAX: 0000000000006d89 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4156.131637] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4156.135871] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4156.140587] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4156.144541] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4156.151167] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4156.156700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4156.159333] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4156.163346] Call Trace: [ 4156.164638] [ 4156.165815] ? show_regs.cold.9+0x22/0x2f [ 4156.167851] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4156.169924] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4156.171883] ? watchdog+0x40/0x40 [ 4156.172973] ? hrtimer_interrupt+0x128/0x2c0 [ 4156.174820] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4156.176450] ? apic_timer_interrupt+0xf/0x20 [ 4156.178912] [ 4156.179665] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4156.183178] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4156.186341] do_raw_spin_lock+0x177/0x190 [ 4156.188303] _raw_spin_lock+0x21/0x30 [ 4156.189869] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4156.192796] process_one_work+0x2c8/0x700 [ 4156.194501] worker_thread+0x296/0x6e0 [ 4156.197051] ? rescuer_thread+0x570/0x570 [ 4156.198524] kthread+0x1d1/0x200 [ 4156.199959] ? set_kthread_struct+0x70/0x70 [ 4156.202343] ret_from_fork+0x1f/0x30 [ 4168.057073] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4168.060011] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4168.060865] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) [ 4168.071360] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4168.071436] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4168.071444] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4168.071447] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4168.071461] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4168.071465] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4168.071470] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4168.071472] RDX: 00000000000045da RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4168.071474] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4168.071476] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4168.071478] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4168.071481] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4168.071484] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4168.071486] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4168.071495] Call Trace: [ 4168.071501] [ 4168.071506] ? show_regs.cold.9+0x22/0x2f [ 4168.071515] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4168.071522] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4168.071526] ? watchdog+0x40/0x40 [ 4168.071530] ? hrtimer_interrupt+0x128/0x2c0 [ 4168.071534] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4168.071542] ? apic_timer_interrupt+0xf/0x20 [ 4168.071547] [ 4168.071550] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4168.071555] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4168.071560] do_raw_spin_lock+0x177/0x190 [ 4168.071564] _raw_spin_lock+0x21/0x30 [ 4168.071570] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4168.072406] obd_mod_free+0x46/0x140 [ptlrpc] [ 4168.072619] mdc_commit_open+0x91/0xb0 [mdc] [ 4168.072656] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4168.073080] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4168.076947] lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4168.077088] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4168.077094] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4168.077097] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4168.077111] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4168.077115] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4168.077119] RAX: 00000000000035a3 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4168.077122] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4168.077124] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4168.077126] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4168.077128] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4168.077131] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4168.077134] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4168.077136] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4168.077145] Call Trace: [ 4168.077150] [ 4168.077155] ? show_regs.cold.9+0x22/0x2f [ 4168.077163] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4168.077168] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4168.077172] ? watchdog+0x40/0x40 [ 4168.077177] ? hrtimer_interrupt+0x128/0x2c0 [ 4168.077180] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4168.077187] ? apic_timer_interrupt+0xf/0x20 [ 4168.077191] [ 4168.077193] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4168.077197] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4168.077201] do_raw_spin_lock+0x177/0x190 [ 4168.077206] _raw_spin_lock+0x21/0x30 [ 4168.077211] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4168.077462] ? do_raw_spin_unlock+0x75/0x190 [ 4168.077468] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4168.077683] ? do_wait_intr+0xf0/0xf0 [ 4168.077690] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4168.077954] mdc_reint+0x2d/0x110 [mdc] [ 4168.077992] mdc_unlink+0x658/0x870 [mdc] [ 4168.078360] lmv_unlink+0x34d/0xbd0 [lmv] [ 4168.078394] ll_unlink+0x24d/0x7f0 [lustre] [ 4168.114381] after_reply+0xe2c/0x1780 [ptlrpc] [ 4168.114679] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4168.120214] vfs_unlink+0x202/0x330 [ 4168.124862] ? __next_timer_interrupt+0x160/0x160 [ 4168.124885] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4168.127696] do_unlinkat+0x2fa/0x460 [ 4168.137949] ? do_wait_intr+0xf0/0xf0 [ 4168.141583] __x64_sys_unlink+0x37/0x50 [ 4168.141684] do_syscall_64+0xc1/0x440 [ 4168.141696] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4168.141711] RIP: 0033:0x7fe02c11910b [ 4168.141719] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4168.141723] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4168.141729] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4168.141805] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4168.141807] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4168.141809] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4168.141811] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4168.418058] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4168.420949] kthread+0x1d1/0x200 [ 4168.422837] ? set_kthread_struct+0x70/0x70 [ 4168.424933] ret_from_fork+0x1f/0x30 [ 4182.139026] rcu: INFO: rcu_sched self-detected stall on CPU [ 4182.140865] rcu: 1-....: (1364768 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=392460 [ 4182.146950] (t=2040047 jiffies g=278549 q=11529) [ 4182.149369] Sending NMI from CPU 1 to CPUs 0: [ 4182.153035] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 [ 4182.154275] NMI backtrace for cpu 1 [ 4182.163548] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4182.173914] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4182.180341] Call Trace: [ 4182.181396] [ 4182.182403] ? dump_stack+0xbb/0x10e [ 4182.186340] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4182.188579] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4182.190023] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4182.193704] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4182.198433] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4182.201646] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4182.205256] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4182.211219] ? update_process_times+0x6d/0xc0 [ 4182.217448] ? tick_sched_handle.isra.18+0x27/0x90 [ 4182.222048] ? tick_sched_timer+0x53/0xb0 [ 4182.225843] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4182.230072] ? tick_sched_handle.isra.18+0x90/0x90 [ 4182.234303] ? hrtimer_interrupt+0x128/0x2c0 [ 4182.236992] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4182.243453] ? apic_timer_interrupt+0xf/0x20 [ 4182.248276] [ 4182.251277] ? native_safe_halt+0x2e/0x50 [ 4182.253068] ? kvm_wait+0xcd/0xe0 [ 4182.256939] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4182.259505] ? do_raw_spin_lock+0x177/0x190 [ 4182.267009] ? _raw_spin_lock+0x21/0x30 [ 4182.269463] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4182.273459] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4182.276720] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4182.279246] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4182.283850] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4182.289277] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4182.294157] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4182.304578] ? __next_timer_interrupt+0x160/0x160 [ 4182.308030] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4182.309646] ? do_wait_intr+0xf0/0xf0 [ 4182.326069] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4182.327740] ? kthread+0x1d1/0x200 [ 4182.328701] ? set_kthread_struct+0x70/0x70 [ 4182.336271] ? ret_from_fork+0x1f/0x30 [ 4184.064255] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4184.069576] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4184.108861] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4184.117514] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4184.123450] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4184.131256] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4184.134562] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4184.150436] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4184.158336] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4184.160307] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4184.164219] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4184.166100] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4184.173972] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4184.180831] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4184.188079] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4184.192473] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4184.198563] Call Trace: [ 4184.200821] [ 4184.202474] ? show_regs.cold.9+0x22/0x2f [ 4184.204330] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4184.208309] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4184.210995] ? watchdog+0x40/0x40 [ 4184.212586] ? hrtimer_interrupt+0x128/0x2c0 [ 4184.216592] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4184.220209] ? apic_timer_interrupt+0xf/0x20 [ 4184.222762] [ 4184.223880] ? native_safe_halt+0x2e/0x50 [ 4184.226166] kvm_wait+0xcd/0xe0 [ 4184.230870] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4184.232729] do_raw_spin_lock+0x177/0x190 [ 4184.241387] _raw_spin_lock+0x21/0x30 [ 4184.243403] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4184.248425] process_one_work+0x2c8/0x700 [ 4184.251214] worker_thread+0x296/0x6e0 [ 4184.252522] ? rescuer_thread+0x570/0x570 [ 4184.254298] kthread+0x1d1/0x200 [ 4184.255580] ? set_kthread_struct+0x70/0x70 [ 4184.258570] ret_from_fork+0x1f/0x30 [ 4208.057026] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4208.059508] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4208.060021] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4208.078766] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4208.078780] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4208.078783] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4208.078798] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4208.078856] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 [ 4208.083441] Modules linked in: [ 4208.089775] ORIG_RAX: ffffffffffffff13 [ 4208.089785] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4208.089787] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4208.089789] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4208.089791] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4208.089793] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4208.089795] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4208.089798] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4208.089800] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4208.089808] Call Trace: [ 4208.089815] [ 4208.089821] ? show_regs.cold.9+0x22/0x2f [ 4208.089832] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4208.089838] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4208.089841] ? watchdog+0x40/0x40 [ 4208.089845] ? hrtimer_interrupt+0x128/0x2c0 [ 4208.089848] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4208.089855] ? apic_timer_interrupt+0xf/0x20 [ 4208.089859] [ 4208.089861] ? native_safe_halt+0x2e/0x50 [ 4208.089866] kvm_wait+0xcd/0xe0 [ 4208.089873] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4208.089881] do_raw_spin_lock+0x177/0x190 [ 4208.089886] _raw_spin_lock+0x21/0x30 [ 4208.089892] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4208.090170] ? do_raw_spin_unlock+0x75/0x190 [ 4208.090175] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4208.099837] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4208.099918] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4208.099926] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4208.099928] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4208.099941] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4208.099945] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4208.099949] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4208.099951] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4208.099953] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4208.099955] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4208.099958] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4208.099961] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4208.099963] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4208.099966] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4208.099974] Call Trace: [ 4208.099979] [ 4208.099986] ? show_regs.cold.9+0x22/0x2f [ 4208.099995] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4208.100002] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4208.100007] ? watchdog+0x40/0x40 [ 4208.100011] ? hrtimer_interrupt+0x128/0x2c0 [ 4208.100015] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4208.100022] ? apic_timer_interrupt+0xf/0x20 [ 4208.100026] [ 4208.100029] ? native_safe_halt+0x2e/0x50 [ 4208.100032] kvm_wait+0xcd/0xe0 [ 4208.100040] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4208.100048] do_raw_spin_lock+0x177/0x190 [ 4208.100052] _raw_spin_lock+0x21/0x30 [ 4208.100057] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4208.100326] obd_mod_free+0x46/0x140 [ptlrpc] [ 4208.100527] mdc_commit_open+0x91/0xb0 [mdc] [ 4208.100563] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4208.100759] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4208.100976] after_reply+0xe2c/0x1780 [ptlrpc] [ 4208.101169] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4208.101365] ? __next_timer_interrupt+0x160/0x160 [ 4208.101374] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4208.101572] ? do_wait_intr+0xf0/0xf0 [ 4208.101580] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4208.101776] kthread+0x1d1/0x200 [ 4208.101782] ? set_kthread_struct+0x70/0x70 [ 4208.101785] ret_from_fork+0x1f/0x30 [ 4208.265604] ? do_wait_intr+0xf0/0xf0 [ 4208.266798] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4208.268573] mdc_reint+0x2d/0x110 [mdc] [ 4208.269873] mdc_unlink+0x658/0x870 [mdc] [ 4208.271200] lmv_unlink+0x34d/0xbd0 [lmv] [ 4208.272568] ll_unlink+0x24d/0x7f0 [lustre] [ 4208.274217] vfs_unlink+0x202/0x330 [ 4208.275694] do_unlinkat+0x2fa/0x460 [ 4208.277019] __x64_sys_unlink+0x37/0x50 [ 4208.278377] do_syscall_64+0xc1/0x440 [ 4208.279798] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4208.281853] RIP: 0033:0x7fe02c11910b [ 4208.283025] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4208.288766] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4208.292538] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4208.294776] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4208.297124] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4208.299982] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4208.302486] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4212.064129] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4212.072677] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4212.099775] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4212.104652] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4212.109617] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4212.113819] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4212.116855] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4212.128513] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4212.131745] RAX: 0000000000007e8f RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4212.135674] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4212.140205] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4212.144274] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4212.150130] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4212.153843] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4212.159317] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4212.164092] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4212.167819] Call Trace: [ 4212.170393] [ 4212.171456] ? show_regs.cold.9+0x22/0x2f [ 4212.174077] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4212.177071] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4212.179071] ? watchdog+0x40/0x40 [ 4212.181273] ? hrtimer_interrupt+0x128/0x2c0 [ 4212.183026] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4212.185091] ? apic_timer_interrupt+0xf/0x20 [ 4212.187497] [ 4212.188923] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4212.191151] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4212.193905] do_raw_spin_lock+0x177/0x190 [ 4212.195724] _raw_spin_lock+0x21/0x30 [ 4212.197706] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4212.199889] process_one_work+0x2c8/0x700 [ 4212.201323] worker_thread+0x296/0x6e0 [ 4212.204885] ? rescuer_thread+0x570/0x570 [ 4212.208627] kthread+0x1d1/0x200 [ 4212.210064] ? set_kthread_struct+0x70/0x70 [ 4212.212021] ret_from_fork+0x1f/0x30 [ 4236.057141] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4236.060838] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4236.061887] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4236.061950] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4236.061954] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4236.061956] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4236.061968] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4236.061971] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4236.061974] RAX: 0000000000007e29 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4236.061976] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4236.061977] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4236.061978] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4236.061980] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4236.061983] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4236.061986] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4236.061988] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4236.062079] Call Trace: [ 4236.062084] [ 4236.062089] ? show_regs.cold.9+0x22/0x2f [ 4236.062096] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4236.062101] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4236.062103] ? watchdog+0x40/0x40 [ 4236.062106] ? hrtimer_interrupt+0x128/0x2c0 [ 4236.062109] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4236.062114] ? apic_timer_interrupt+0xf/0x20 [ 4236.062117] [ 4236.062119] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4236.062121] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4236.062124] do_raw_spin_lock+0x177/0x190 [ 4236.062127] _raw_spin_lock+0x21/0x30 [ 4236.062131] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4236.062380] ? do_raw_spin_unlock+0x75/0x190 [ 4236.062383] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4236.064983] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) [ 4236.086357] ? do_wait_intr+0xf0/0xf0 [ 4236.086375] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4236.086667] mdc_reint+0x2d/0x110 [mdc] [ 4236.086709] mdc_unlink+0x658/0x870 [mdc] [ 4236.086737] lmv_unlink+0x34d/0xbd0 [lmv] [ 4236.086765] ll_unlink+0x24d/0x7f0 [lustre] [ 4236.086886] vfs_unlink+0x202/0x330 [ 4236.086894] do_unlinkat+0x2fa/0x460 [ 4236.086898] __x64_sys_unlink+0x37/0x50 [ 4236.086901] do_syscall_64+0xc1/0x440 [ 4236.086908] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4236.086915] RIP: 0033:0x7fe02c11910b [ 4236.086922] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4236.086926] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4236.086931] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4236.086933] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4236.086935] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4236.086937] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4236.086939] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4236.227397] mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4236.248751] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4236.253486] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4236.257322] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4236.260317] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4236.267103] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4236.270356] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4236.273309] RDX: 00000000000006f6 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4236.275592] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4236.277724] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4236.279957] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4236.282519] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4236.285333] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4236.287586] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4236.289743] Call Trace: [ 4236.290532] [ 4236.291230] ? show_regs.cold.9+0x22/0x2f [ 4236.292435] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4236.294061] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4236.295381] ? watchdog+0x40/0x40 [ 4236.296320] ? hrtimer_interrupt+0x128/0x2c0 [ 4236.298091] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4236.299720] ? apic_timer_interrupt+0xf/0x20 [ 4236.301308] [ 4236.301920] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4236.304072] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4236.306281] do_raw_spin_lock+0x177/0x190 [ 4236.307919] _raw_spin_lock+0x21/0x30 [ 4236.309116] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4236.311250] obd_mod_free+0x46/0x140 [ptlrpc] [ 4236.313383] mdc_commit_open+0x91/0xb0 [mdc] [ 4236.315330] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4236.317690] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4236.320308] after_reply+0xe2c/0x1780 [ptlrpc] [ 4236.322277] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4236.324720] ? __next_timer_interrupt+0x160/0x160 [ 4236.326980] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4236.328943] ? do_wait_intr+0xf0/0xf0 [ 4236.330390] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4236.332952] kthread+0x1d1/0x200 [ 4236.334246] ? set_kthread_struct+0x70/0x70 [ 4236.335942] ret_from_fork+0x1f/0x30 [ 4240.066554] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4240.068634] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4240.082940] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4240.086138] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4240.089311] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4240.091675] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4240.093755] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4240.099961] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4240.102051] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4240.104678] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4240.106985] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4240.109537] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4240.111608] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4240.113634] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4240.115859] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4240.117977] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4240.119932] Call Trace: [ 4240.120584] [ 4240.121330] ? show_regs.cold.9+0x22/0x2f [ 4240.122577] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4240.123996] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4240.125199] ? watchdog+0x40/0x40 [ 4240.126081] ? hrtimer_interrupt+0x128/0x2c0 [ 4240.127508] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4240.128681] ? apic_timer_interrupt+0xf/0x20 [ 4240.129925] [ 4240.130661] ? native_safe_halt+0x2e/0x50 [ 4240.131754] kvm_wait+0xcd/0xe0 [ 4240.132850] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4240.134650] do_raw_spin_lock+0x177/0x190 [ 4240.135772] _raw_spin_lock+0x21/0x30 [ 4240.137010] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4240.138830] process_one_work+0x2c8/0x700 [ 4240.140010] worker_thread+0x296/0x6e0 [ 4240.141201] ? rescuer_thread+0x570/0x570 [ 4240.142260] kthread+0x1d1/0x200 [ 4240.143447] ? set_kthread_struct+0x70/0x70 [ 4240.144767] ret_from_fork+0x1f/0x30 [ 4264.060605] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4264.061227] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4264.064557] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4264.064675] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4264.064683] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4264.064686] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4264.065212] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4264.065216] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4264.065221] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4264.065223] RDX: 0000000000007cb9 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4264.065225] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4264.065227] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4264.065230] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4264.065232] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4264.065235] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4264.065237] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4264.065245] Call Trace: [ 4264.065266] [ 4264.065273] ? show_regs.cold.9+0x22/0x2f [ 4264.065282] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4264.065287] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4264.065292] ? watchdog+0x40/0x40 [ 4264.065296] ? hrtimer_interrupt+0x128/0x2c0 [ 4264.065300] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4264.065373] ? apic_timer_interrupt+0xf/0x20 [ 4264.065376] [ 4264.065379] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4264.065383] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4264.065387] do_raw_spin_lock+0x177/0x190 [ 4264.065392] _raw_spin_lock+0x21/0x30 [ 4264.065398] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4264.065932] obd_mod_free+0x46/0x140 [ptlrpc] [ 4264.066983] mdc_commit_open+0x91/0xb0 [mdc] [ 4264.067025] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4264.074399] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4264.074478] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4264.074485] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4264.074489] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4264.074504] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4264.074508] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 4264.074513] RAX: 000000000000173a RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4264.074516] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4264.074518] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4264.074520] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4264.074523] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4264.074527] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4264.074529] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4264.074531] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4264.074540] Call Trace: [ 4264.074546] [ 4264.074553] ? show_regs.cold.9+0x22/0x2f [ 4264.074564] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4264.074570] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4264.074575] ? watchdog+0x40/0x40 [ 4264.074579] ? hrtimer_interrupt+0x128/0x2c0 [ 4264.074583] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4264.074592] ? apic_timer_interrupt+0xf/0x20 [ 4264.074596] [ 4264.074599] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4264.074603] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4264.074608] do_raw_spin_lock+0x177/0x190 [ 4264.074612] _raw_spin_lock+0x21/0x30 [ 4264.074617] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4264.074953] ? do_raw_spin_unlock+0x75/0x190 [ 4264.074959] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4264.075155] ? do_wait_intr+0xf0/0xf0 [ 4264.075163] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4264.075704] mdc_reint+0x2d/0x110 [mdc] [ 4264.075748] mdc_unlink+0x658/0x870 [mdc] [ 4264.075779] lmv_unlink+0x34d/0xbd0 [lmv] [ 4264.075812] ll_unlink+0x24d/0x7f0 [lustre] [ 4264.111091] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4264.117915] vfs_unlink+0x202/0x330 [ 4264.117949] do_unlinkat+0x2fa/0x460 [ 4264.117955] __x64_sys_unlink+0x37/0x50 [ 4264.117959] do_syscall_64+0xc1/0x440 [ 4264.117966] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4264.117975] RIP: 0033:0x7fe02c11910b [ 4264.117983] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4264.117987] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4264.117992] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4264.117995] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4264.117997] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4264.117999] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4264.118001] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4264.513496] after_reply+0xe2c/0x1780 [ptlrpc] [ 4264.515028] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4264.523811] ? __next_timer_interrupt+0x160/0x160 [ 4264.527610] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4264.533656] ? do_wait_intr+0xf0/0xf0 [ 4264.535254] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4264.540280] kthread+0x1d1/0x200 [ 4264.544005] ? set_kthread_struct+0x70/0x70 [ 4264.547753] ret_from_fork+0x1f/0x30 [ 4268.064029] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4268.066641] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4268.134368] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4268.142727] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4268.148591] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4268.157374] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4268.161331] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4268.187907] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4268.196436] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4268.207951] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4268.215223] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4268.223271] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4268.225449] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4268.230372] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4268.236524] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4268.241700] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4268.252186] Call Trace: [ 4268.252903] [ 4268.253550] ? show_regs.cold.9+0x22/0x2f [ 4268.256218] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4268.258789] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4268.261226] ? watchdog+0x40/0x40 [ 4268.263205] ? hrtimer_interrupt+0x128/0x2c0 [ 4268.265570] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4268.267814] ? apic_timer_interrupt+0xf/0x20 [ 4268.270419] [ 4268.271678] ? native_safe_halt+0x2e/0x50 [ 4268.274681] kvm_wait+0xcd/0xe0 [ 4268.276749] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4268.279947] do_raw_spin_lock+0x177/0x190 [ 4268.284339] _raw_spin_lock+0x21/0x30 [ 4268.286662] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4268.292534] process_one_work+0x2c8/0x700 [ 4268.294753] worker_thread+0x296/0x6e0 [ 4268.296951] ? rescuer_thread+0x570/0x570 [ 4268.299412] kthread+0x1d1/0x200 [ 4268.300871] ? set_kthread_struct+0x70/0x70 [ 4268.303418] ret_from_fork+0x1f/0x30 [ 4292.057197] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4292.060094] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4292.060181] Modules linked in: [ 4292.062966] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4292.063049] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4292.063058] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4292.063061] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4292.064794] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4292.064883] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4292.064893] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4292.064898] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4292.064909] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4292.064916] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4292.064925] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4292.064927] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4292.064930] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4292.064932] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4292.064934] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4292.064937] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4292.064940] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4292.064942] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4292.064950] Call Trace: [ 4292.064955] [ 4292.064962] ? show_regs.cold.9+0x22/0x2f [ 4292.064972] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4292.064987] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4292.064995] ? watchdog+0x40/0x40 [ 4292.065003] ? hrtimer_interrupt+0x128/0x2c0 [ 4292.065010] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4292.065025] ? apic_timer_interrupt+0xf/0x20 [ 4292.065033] [ 4292.065036] ? native_safe_halt+0x2e/0x50 [ 4292.065040] kvm_wait+0xcd/0xe0 [ 4292.065052] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4292.065060] do_raw_spin_lock+0x177/0x190 [ 4292.065064] _raw_spin_lock+0x21/0x30 [ 4292.065069] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4292.065390] ? do_raw_spin_unlock+0x75/0x190 [ 4292.065397] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4292.088965] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4292.095611] ? do_wait_intr+0xf0/0xf0 [ 4292.098962] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4292.098974] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4292.098976] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4292.098978] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4292.098981] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4292.098983] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4292.098986] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4292.098989] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4292.098991] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4292.098999] Call Trace: [ 4292.099004] [ 4292.099009] ? show_regs.cold.9+0x22/0x2f [ 4292.099023] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4292.099030] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4292.099034] ? watchdog+0x40/0x40 [ 4292.099039] ? hrtimer_interrupt+0x128/0x2c0 [ 4292.099043] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4292.099050] ? apic_timer_interrupt+0xf/0x20 [ 4292.099055] [ 4292.099057] ? native_safe_halt+0x2e/0x50 [ 4292.099061] kvm_wait+0xcd/0xe0 [ 4292.099068] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4292.099075] do_raw_spin_lock+0x177/0x190 [ 4292.099079] _raw_spin_lock+0x21/0x30 [ 4292.099085] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4292.099425] obd_mod_free+0x46/0x140 [ptlrpc] [ 4292.101101] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4292.124532] mdc_commit_open+0x91/0xb0 [mdc] [ 4292.127704] mdc_reint+0x2d/0x110 [mdc] [ 4292.130187] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4292.131474] mdc_unlink+0x658/0x870 [mdc] [ 4292.136556] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4292.138635] lmv_unlink+0x34d/0xbd0 [lmv] [ 4292.140353] after_reply+0xe2c/0x1780 [ptlrpc] [ 4292.142955] ll_unlink+0x24d/0x7f0 [lustre] [ 4292.143077] vfs_unlink+0x202/0x330 [ 4292.143088] do_unlinkat+0x2fa/0x460 [ 4292.143093] __x64_sys_unlink+0x37/0x50 [ 4292.143097] do_syscall_64+0xc1/0x440 [ 4292.143104] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4292.143113] RIP: 0033:0x7fe02c11910b [ 4292.143121] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4292.143125] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4292.143131] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4292.143133] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4292.143136] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4292.143138] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4292.143140] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4292.461715] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4292.467507] ? __next_timer_interrupt+0x160/0x160 [ 4292.470462] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4292.475054] ? do_wait_intr+0xf0/0xf0 [ 4292.476971] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4292.479337] kthread+0x1d1/0x200 [ 4292.480402] ? set_kthread_struct+0x70/0x70 [ 4292.483310] ret_from_fork+0x1f/0x30 [ 4296.064389] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4296.076664] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4296.101465] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4296.114951] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4296.121653] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4296.126480] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4296.131557] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4296.142019] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4296.145836] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4296.149047] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4296.159624] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4296.164050] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4296.170465] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4296.175900] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4296.180492] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4296.183371] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4296.186829] Call Trace: [ 4296.188182] [ 4296.189414] ? show_regs.cold.9+0x22/0x2f [ 4296.191713] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4296.200449] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4296.202975] ? watchdog+0x40/0x40 [ 4296.207882] ? hrtimer_interrupt+0x128/0x2c0 [ 4296.209234] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4296.213391] ? apic_timer_interrupt+0xf/0x20 [ 4296.215660] [ 4296.217341] ? native_safe_halt+0x2e/0x50 [ 4296.219301] kvm_wait+0xcd/0xe0 [ 4296.224633] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4296.226490] do_raw_spin_lock+0x177/0x190 [ 4296.227964] _raw_spin_lock+0x21/0x30 [ 4296.229818] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4296.233675] process_one_work+0x2c8/0x700 [ 4296.235957] worker_thread+0x296/0x6e0 [ 4296.237471] ? rescuer_thread+0x570/0x570 [ 4296.239564] kthread+0x1d1/0x200 [ 4296.240670] ? set_kthread_struct+0x70/0x70 [ 4296.243368] ret_from_fork+0x1f/0x30 [ 4320.057127] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4320.061445] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4320.064645] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4320.064806] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4320.064817] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4320.064820] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4320.064834] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4320.064838] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4320.064842] RAX: 00000000000078de RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4320.064845] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4320.064847] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4320.064849] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4320.064851] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4320.064855] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4320.064857] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4320.064860] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4320.064868] Call Trace: [ 4320.064873] [ 4320.064878] ? show_regs.cold.9+0x22/0x2f [ 4320.064888] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4320.064894] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4320.064898] ? watchdog+0x40/0x40 [ 4320.064902] ? hrtimer_interrupt+0x128/0x2c0 [ 4320.064906] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4320.064915] ? apic_timer_interrupt+0xf/0x20 [ 4320.064919] [ 4320.064922] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4320.064927] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4320.064932] do_raw_spin_lock+0x177/0x190 [ 4320.064949] _raw_spin_lock+0x21/0x30 [ 4320.064956] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4320.070784] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4320.070887] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4320.070894] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4320.070896] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4320.070913] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4320.070917] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4320.070922] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4320.070924] RDX: 0000000000006f21 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4320.070926] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4320.070928] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4320.070930] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4320.070933] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4320.070935] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4320.070949] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4320.070957] Call Trace: [ 4320.070962] [ 4320.070968] ? show_regs.cold.9+0x22/0x2f [ 4320.070986] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4320.070998] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4320.071005] ? watchdog+0x40/0x40 [ 4320.071009] ? hrtimer_interrupt+0x128/0x2c0 [ 4320.071013] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4320.071022] ? apic_timer_interrupt+0xf/0x20 [ 4320.071027] [ 4320.071031] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4320.071037] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4320.071042] do_raw_spin_lock+0x177/0x190 [ 4320.071050] _raw_spin_lock+0x21/0x30 [ 4320.071059] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4320.071486] obd_mod_free+0x46/0x140 [ptlrpc] [ 4320.071835] mdc_commit_open+0x91/0xb0 [mdc] [ 4320.071869] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4320.072157] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4320.072384] after_reply+0xe2c/0x1780 [ptlrpc] [ 4320.072621] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4320.072993] ? __next_timer_interrupt+0x160/0x160 [ 4320.073003] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4320.073216] ? do_wait_intr+0xf0/0xf0 [ 4320.073225] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4320.073555] kthread+0x1d1/0x200 [ 4320.073561] ? set_kthread_struct+0x70/0x70 [ 4320.073569] ret_from_fork+0x1f/0x30 [ 4320.351800] ? do_raw_spin_unlock+0x75/0x190 [ 4320.353302] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4320.355190] ? do_wait_intr+0xf0/0xf0 [ 4320.356521] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4320.358947] mdc_reint+0x2d/0x110 [mdc] [ 4320.360621] mdc_unlink+0x658/0x870 [mdc] [ 4320.362149] lmv_unlink+0x34d/0xbd0 [lmv] [ 4320.364222] ll_unlink+0x24d/0x7f0 [lustre] [ 4320.366911] vfs_unlink+0x202/0x330 [ 4320.368289] do_unlinkat+0x2fa/0x460 [ 4320.369849] __x64_sys_unlink+0x37/0x50 [ 4320.371234] do_syscall_64+0xc1/0x440 [ 4320.372684] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4320.375423] RIP: 0033:0x7fe02c11910b [ 4320.376863] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4320.384531] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4320.388941] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4320.392912] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4320.398697] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4320.402084] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4320.405629] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4324.064021] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4324.070436] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4324.090068] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4324.094054] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4324.097336] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4324.099973] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4324.102043] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4324.109074] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4324.111696] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4324.113860] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4324.116030] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4324.118179] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4324.120598] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4324.122728] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4324.125632] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4324.127544] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4324.129847] Call Trace: [ 4324.130791] [ 4324.131580] ? show_regs.cold.9+0x22/0x2f [ 4324.132969] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4324.134617] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4324.136549] ? watchdog+0x40/0x40 [ 4324.137365] ? hrtimer_interrupt+0x128/0x2c0 [ 4324.139141] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4324.140730] ? apic_timer_interrupt+0xf/0x20 [ 4324.143494] [ 4324.144067] ? native_safe_halt+0x2e/0x50 [ 4324.147372] kvm_wait+0xcd/0xe0 [ 4324.148436] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4324.151044] do_raw_spin_lock+0x177/0x190 [ 4324.152505] _raw_spin_lock+0x21/0x30 [ 4324.154511] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4324.157207] process_one_work+0x2c8/0x700 [ 4324.159023] worker_thread+0x296/0x6e0 [ 4324.160586] ? rescuer_thread+0x570/0x570 [ 4324.161994] kthread+0x1d1/0x200 [ 4324.163103] ? set_kthread_struct+0x70/0x70 [ 4324.164726] ret_from_fork+0x1f/0x30 [ 4348.057116] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4348.060371] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4348.061143] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) [ 4348.063876] Modules linked in: [ 4348.063876] ptlrpc(O) obdclass(O) ksocklnd(O) [ 4348.069857] lustre(O) [ 4348.071569] lnet(O) [ 4348.073322] osp(O) [ 4348.075265] libcfs(O) [ 4348.076223] ofd(O) [ 4348.077383] rpcsec_gss_krb5 [ 4348.078334] lod(O) [ 4348.089767] auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 [ 4348.091381] mdt(O) [ 4348.092788] sha512_generic [ 4348.102084] mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common [ 4348.104402] [ 4348.105562] sb_edac [ 4348.127431] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4348.127933] rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4348.127953] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4348.127956] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4348.127958] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4348.127970] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4348.127973] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4348.127976] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4348.127978] RDX: 00000000000051e3 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4348.127980] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4348.127981] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4348.127983] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4348.127985] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4348.127987] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4348.127989] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4348.127996] Call Trace: [ 4348.128000] [ 4348.128004] ? show_regs.cold.9+0x22/0x2f [ 4348.128010] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4348.128015] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4348.128018] ? watchdog+0x40/0x40 [ 4348.128021] ? hrtimer_interrupt+0x128/0x2c0 [ 4348.128024] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4348.128032] ? apic_timer_interrupt+0xf/0x20 [ 4348.128035] [ 4348.128037] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4348.128040] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4348.128043] do_raw_spin_lock+0x177/0x190 [ 4348.128046] _raw_spin_lock+0x21/0x30 [ 4348.128051] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4348.128319] obd_mod_free+0x46/0x140 [ptlrpc] [ 4348.128479] mdc_commit_open+0x91/0xb0 [mdc] [ 4348.128510] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4348.128634] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4348.128856] after_reply+0xe2c/0x1780 [ptlrpc] [ 4348.129663] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4348.133793] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4348.134221] ? __next_timer_interrupt+0x160/0x160 [ 4348.134237] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4348.134655] ? do_wait_intr+0xf0/0xf0 [ 4348.134664] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4348.134883] kthread+0x1d1/0x200 [ 4348.134889] ? set_kthread_struct+0x70/0x70 [ 4348.134893] ret_from_fork+0x1f/0x30 [ 4348.268049] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4348.270909] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4348.278602] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4348.283972] RAX: 0000000000000b16 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4348.287815] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4348.292142] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4348.296305] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4348.303480] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4348.305972] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4348.309921] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4348.316692] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4348.322052] Call Trace: [ 4348.323571] [ 4348.325217] ? show_regs.cold.9+0x22/0x2f [ 4348.329629] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4348.331678] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4348.333481] ? watchdog+0x40/0x40 [ 4348.334944] ? hrtimer_interrupt+0x128/0x2c0 [ 4348.336525] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4348.338316] ? apic_timer_interrupt+0xf/0x20 [ 4348.339928] [ 4348.341120] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4348.343418] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4348.345619] do_raw_spin_lock+0x177/0x190 [ 4348.347477] _raw_spin_lock+0x21/0x30 [ 4348.348971] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4348.351282] ? do_raw_spin_unlock+0x75/0x190 [ 4348.357097] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4348.360355] ? do_wait_intr+0xf0/0xf0 [ 4348.361709] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4348.371176] mdc_reint+0x2d/0x110 [mdc] [ 4348.377058] mdc_unlink+0x658/0x870 [mdc] [ 4348.381567] lmv_unlink+0x34d/0xbd0 [lmv] [ 4348.383801] ll_unlink+0x24d/0x7f0 [lustre] [ 4348.386268] vfs_unlink+0x202/0x330 [ 4348.389387] do_unlinkat+0x2fa/0x460 [ 4348.394217] __x64_sys_unlink+0x37/0x50 [ 4348.395610] do_syscall_64+0xc1/0x440 [ 4348.397681] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4348.400289] RIP: 0033:0x7fe02c11910b [ 4348.401584] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4348.413789] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4348.421295] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4348.425553] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4348.429396] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4348.434298] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4348.437986] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4352.064031] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4352.067783] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4352.090971] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4352.096636] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4352.100045] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4352.103079] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4352.105918] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4352.117225] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4352.121293] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4352.125344] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4352.129066] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4352.132264] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4352.135966] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4352.139562] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4352.143559] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4352.146608] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4352.150004] Call Trace: [ 4352.151092] [ 4352.152164] ? show_regs.cold.9+0x22/0x2f [ 4352.154888] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4352.157089] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4352.158620] ? watchdog+0x40/0x40 [ 4352.160165] ? hrtimer_interrupt+0x128/0x2c0 [ 4352.162269] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4352.164992] ? apic_timer_interrupt+0xf/0x20 [ 4352.167036] [ 4352.168215] ? native_safe_halt+0x2e/0x50 [ 4352.170109] kvm_wait+0xcd/0xe0 [ 4352.171519] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4352.173970] do_raw_spin_lock+0x177/0x190 [ 4352.176102] _raw_spin_lock+0x21/0x30 [ 4352.177264] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4352.179276] process_one_work+0x2c8/0x700 [ 4352.180767] worker_thread+0x296/0x6e0 [ 4352.182102] ? rescuer_thread+0x570/0x570 [ 4352.183829] kthread+0x1d1/0x200 [ 4352.184991] ? set_kthread_struct+0x70/0x70 [ 4352.186619] ret_from_fork+0x1f/0x30 [ 4362.142041] rcu: INFO: rcu_sched self-detected stall on CPU [ 4362.143527] rcu: 1-....: (1508053 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=429926 [ 4362.146165] (t=2220050 jiffies g=278549 q=12286) [ 4362.147366] Sending NMI from CPU 1 to CPUs 0: [ 4362.148185] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 [ 4362.149164] NMI backtrace for cpu 1 [ 4362.150345] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4362.153745] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4362.155802] Call Trace: [ 4362.156398] [ 4362.156791] ? dump_stack+0xbb/0x10e [ 4362.157748] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4362.159162] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4362.160438] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4362.161975] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4362.163692] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4362.165365] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4362.166677] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4362.168690] ? update_process_times+0x6d/0xc0 [ 4362.170355] ? tick_sched_handle.isra.18+0x27/0x90 [ 4362.173246] ? tick_sched_timer+0x53/0xb0 [ 4362.174905] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4362.178089] ? tick_sched_handle.isra.18+0x90/0x90 [ 4362.181906] ? hrtimer_interrupt+0x128/0x2c0 [ 4362.183882] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4362.186408] ? apic_timer_interrupt+0xf/0x20 [ 4362.188362] [ 4362.189087] ? native_safe_halt+0x2e/0x50 [ 4362.190678] ? kvm_wait+0xcd/0xe0 [ 4362.192212] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4362.194469] ? do_raw_spin_lock+0x177/0x190 [ 4362.196315] ? _raw_spin_lock+0x21/0x30 [ 4362.198222] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4362.200850] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4362.203417] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4362.205415] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4362.207714] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4362.212134] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4362.214667] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4362.219909] ? __next_timer_interrupt+0x160/0x160 [ 4362.222756] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4362.224985] ? do_wait_intr+0xf0/0xf0 [ 4362.228361] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4362.231977] ? kthread+0x1d1/0x200 [ 4362.233425] ? set_kthread_struct+0x70/0x70 [ 4362.235214] ? ret_from_fork+0x1f/0x30 [ 4380.068282] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4380.078467] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4380.124485] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4380.133412] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4380.138861] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4380.143481] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4380.149242] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4380.178366] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4380.185067] RAX: 0000000000007bbd RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4380.195059] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4380.204960] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4380.218362] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4380.232849] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4380.237174] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4380.243033] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4380.246434] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4380.253502] Call Trace: [ 4380.256412] [ 4380.257222] ? show_regs.cold.9+0x22/0x2f [ 4380.262380] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4380.265462] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4380.267661] ? watchdog+0x40/0x40 [ 4380.273108] ? hrtimer_interrupt+0x128/0x2c0 [ 4380.278041] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4380.281823] ? apic_timer_interrupt+0xf/0x20 [ 4380.285086] [ 4380.286953] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4380.293429] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4380.302910] do_raw_spin_lock+0x177/0x190 [ 4380.308503] _raw_spin_lock+0x21/0x30 [ 4380.311083] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4380.315438] process_one_work+0x2c8/0x700 [ 4380.318828] worker_thread+0x296/0x6e0 [ 4380.320976] ? rescuer_thread+0x570/0x570 [ 4380.322769] kthread+0x1d1/0x200 [ 4380.324118] ? set_kthread_struct+0x70/0x70 [ 4380.326507] ret_from_fork+0x1f/0x30 [ 4388.059427] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4388.060564] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4388.064622] Modules linked in: lustre(O) osp(O) ofd(O) [ 4388.069514] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4388.069590] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4388.069598] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4388.069600] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4388.069614] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4388.069618] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4388.069623] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4388.069626] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4388.069628] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4388.069630] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4388.069633] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4388.069637] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4388.069639] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4388.069642] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4388.069703] Call Trace: [ 4388.069708] [ 4388.069713] ? show_regs.cold.9+0x22/0x2f [ 4388.069723] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4388.069729] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4388.069733] ? watchdog+0x40/0x40 [ 4388.069737] ? hrtimer_interrupt+0x128/0x2c0 [ 4388.069742] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4388.069759] ? apic_timer_interrupt+0xf/0x20 [ 4388.069764] [ 4388.069767] ? native_safe_halt+0x2e/0x50 [ 4388.069771] kvm_wait+0xcd/0xe0 [ 4388.069779] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4388.069786] do_raw_spin_lock+0x177/0x190 [ 4388.069790] _raw_spin_lock+0x21/0x30 [ 4388.069795] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4388.070266] obd_mod_free+0x46/0x140 [ptlrpc] [ 4388.070609] mdc_commit_open+0x91/0xb0 [mdc] [ 4388.070691] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4388.070905] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4388.071427] after_reply+0xe2c/0x1780 [ptlrpc] [ 4388.071633] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4388.071947] ? __next_timer_interrupt+0x160/0x160 [ 4388.071957] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4388.074849] lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4388.074917] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4388.074923] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4388.074926] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4388.074939] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4388.074942] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4388.074947] RAX: 0000000000007f91 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4388.074949] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4388.074951] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4388.074953] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4388.074955] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4388.074958] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4388.074961] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4388.106775] ? do_wait_intr+0xf0/0xf0 [ 4388.106796] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4388.107176] kthread+0x1d1/0x200 [ 4388.107182] ? set_kthread_struct+0x70/0x70 [ 4388.107187] ret_from_fork+0x1f/0x30 [ 4388.377967] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4388.384308] Call Trace: [ 4388.385334] [ 4388.386487] ? show_regs.cold.9+0x22/0x2f [ 4388.388722] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4388.391420] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4388.395326] ? watchdog+0x40/0x40 [ 4388.398301] ? hrtimer_interrupt+0x128/0x2c0 [ 4388.403927] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4388.409529] ? apic_timer_interrupt+0xf/0x20 [ 4388.416445] [ 4388.417270] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4388.419102] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4388.423644] do_raw_spin_lock+0x177/0x190 [ 4388.426526] _raw_spin_lock+0x21/0x30 [ 4388.428567] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4388.432473] ? do_raw_spin_unlock+0x75/0x190 [ 4388.435140] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4388.437084] ? do_wait_intr+0xf0/0xf0 [ 4388.439255] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4388.443435] mdc_reint+0x2d/0x110 [mdc] [ 4388.446783] mdc_unlink+0x658/0x870 [mdc] [ 4388.448502] lmv_unlink+0x34d/0xbd0 [lmv] [ 4388.449987] ll_unlink+0x24d/0x7f0 [lustre] [ 4388.457920] vfs_unlink+0x202/0x330 [ 4388.460568] do_unlinkat+0x2fa/0x460 [ 4388.464070] __x64_sys_unlink+0x37/0x50 [ 4388.468109] do_syscall_64+0xc1/0x440 [ 4388.469976] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4388.475620] RIP: 0033:0x7fe02c11910b [ 4388.479489] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4388.499576] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4388.502521] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4388.509553] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4388.517351] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4388.525936] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4388.534223] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4408.064094] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4408.066215] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4408.121715] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4408.134753] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4408.140019] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4408.148794] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4408.151308] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4408.166457] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4408.171377] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4408.176429] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4408.184287] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4408.196435] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4408.201373] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4408.204916] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4408.210743] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4408.213202] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4408.218597] Call Trace: [ 4408.221559] [ 4408.227126] ? show_regs.cold.9+0x22/0x2f [ 4408.233321] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4408.236337] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4408.240063] ? watchdog+0x40/0x40 [ 4408.241387] ? hrtimer_interrupt+0x128/0x2c0 [ 4408.247630] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4408.255664] ? apic_timer_interrupt+0xf/0x20 [ 4408.262441] [ 4408.263632] ? native_safe_halt+0x2e/0x50 [ 4408.270354] kvm_wait+0xcd/0xe0 [ 4408.272522] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4408.276831] do_raw_spin_lock+0x177/0x190 [ 4408.279715] _raw_spin_lock+0x21/0x30 [ 4408.282460] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4408.288149] process_one_work+0x2c8/0x700 [ 4408.291545] worker_thread+0x296/0x6e0 [ 4408.295896] ? rescuer_thread+0x570/0x570 [ 4408.297130] kthread+0x1d1/0x200 [ 4408.298664] ? set_kthread_struct+0x70/0x70 [ 4408.302883] ret_from_fork+0x1f/0x30 [ 4416.058086] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4416.061854] Modules linked in: lustre(O) [ 4416.061995] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4416.064964] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4416.065506] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4416.065513] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4416.065515] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4416.065529] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4416.065533] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4416.065538] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4416.065540] RDX: 0000000000001b4a RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4416.065542] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4416.065544] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4416.065546] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4416.065549] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4416.065551] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4416.065553] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4416.065563] Call Trace: [ 4416.065567] [ 4416.065572] ? show_regs.cold.9+0x22/0x2f [ 4416.065581] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4416.065587] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4416.065591] ? watchdog+0x40/0x40 [ 4416.065595] ? hrtimer_interrupt+0x128/0x2c0 [ 4416.067404] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4416.067414] ? apic_timer_interrupt+0xf/0x20 [ 4416.067419] [ 4416.067422] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4416.067428] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4416.067432] do_raw_spin_lock+0x177/0x190 [ 4416.067437] _raw_spin_lock+0x21/0x30 [ 4416.067443] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4416.067705] obd_mod_free+0x46/0x140 [ptlrpc] [ 4416.067900] mdc_commit_open+0x91/0xb0 [mdc] [ 4416.067937] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4416.068136] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4416.068445] after_reply+0xe2c/0x1780 [ptlrpc] [ 4416.068636] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4416.068834] ? __next_timer_interrupt+0x160/0x160 [ 4416.068845] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4416.069064] ? do_wait_intr+0xf0/0xf0 [ 4416.069072] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4416.072512] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4416.072662] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4416.072668] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4416.072670] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4416.072683] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4416.072687] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4416.072691] RAX: 000000000000348f RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4416.072694] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4416.072696] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4416.072698] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4416.072701] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4416.072704] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4416.072707] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4416.072709] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4416.072717] Call Trace: [ 4416.072721] [ 4416.072726] ? show_regs.cold.9+0x22/0x2f [ 4416.072734] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4416.072740] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4416.072744] ? watchdog+0x40/0x40 [ 4416.072749] ? hrtimer_interrupt+0x128/0x2c0 [ 4416.107608] kthread+0x1d1/0x200 [ 4416.107626] ? set_kthread_struct+0x70/0x70 [ 4416.107630] ret_from_fork+0x1f/0x30 [ 4416.338902] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4416.341681] ? apic_timer_interrupt+0xf/0x20 [ 4416.343965] [ 4416.345059] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4416.350003] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4416.353137] do_raw_spin_lock+0x177/0x190 [ 4416.358821] _raw_spin_lock+0x21/0x30 [ 4416.360458] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4416.365263] ? do_raw_spin_unlock+0x75/0x190 [ 4416.370767] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4416.373852] ? do_wait_intr+0xf0/0xf0 [ 4416.380071] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4416.384083] mdc_reint+0x2d/0x110 [mdc] [ 4416.385858] mdc_unlink+0x658/0x870 [mdc] [ 4416.389508] lmv_unlink+0x34d/0xbd0 [lmv] [ 4416.393566] ll_unlink+0x24d/0x7f0 [lustre] [ 4416.399151] vfs_unlink+0x202/0x330 [ 4416.405024] do_unlinkat+0x2fa/0x460 [ 4416.409081] __x64_sys_unlink+0x37/0x50 [ 4416.416553] do_syscall_64+0xc1/0x440 [ 4416.419856] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4416.426152] RIP: 0033:0x7fe02c11910b [ 4416.430955] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4416.453303] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4416.465605] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4416.474064] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4416.482856] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4416.488640] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4416.495782] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4436.064027] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4436.067872] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4436.147244] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4436.153949] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4436.159474] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4436.166506] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4436.174882] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4436.201208] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4436.206290] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4436.213088] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4436.216266] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4436.223900] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4436.227710] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4436.236640] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4436.243217] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4436.252367] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4436.263433] Call Trace: [ 4436.265799] [ 4436.266578] ? show_regs.cold.9+0x22/0x2f [ 4436.268833] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4436.270860] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4436.280891] ? watchdog+0x40/0x40 [ 4436.284030] ? hrtimer_interrupt+0x128/0x2c0 [ 4436.285101] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4436.286349] ? apic_timer_interrupt+0xf/0x20 [ 4436.287852] [ 4436.288528] ? native_safe_halt+0x2e/0x50 [ 4436.293686] kvm_wait+0xcd/0xe0 [ 4436.294839] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4436.301877] do_raw_spin_lock+0x177/0x190 [ 4436.311746] _raw_spin_lock+0x21/0x30 [ 4436.316536] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4436.319020] process_one_work+0x2c8/0x700 [ 4436.330943] worker_thread+0x296/0x6e0 [ 4436.337929] ? rescuer_thread+0x570/0x570 [ 4436.340621] kthread+0x1d1/0x200 [ 4436.344550] ? set_kthread_struct+0x70/0x70 [ 4436.352370] ret_from_fork+0x1f/0x30 [ 4444.057061] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4444.059671] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4444.060015] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4444.079422] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4444.081390] Modules linked in: lustre(O) [ 4444.085883] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4444.085887] osp(O) ofd(O) lod(O) [ 4444.087076] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4444.090962] mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4444.091036] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4444.091044] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4444.091048] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4444.091062] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4444.091067] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4444.091072] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4444.091074] RDX: 0000000000000237 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4444.091076] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4444.091079] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4444.091082] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4444.091084] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4444.091087] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4444.091090] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4444.091098] Call Trace: [ 4444.091103] [ 4444.091108] ? show_regs.cold.9+0x22/0x2f [ 4444.091117] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4444.091124] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4444.091129] ? watchdog+0x40/0x40 [ 4444.091133] ? hrtimer_interrupt+0x128/0x2c0 [ 4444.091137] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4444.091144] ? apic_timer_interrupt+0xf/0x20 [ 4444.091148] [ 4444.091161] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4444.091166] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4444.091170] do_raw_spin_lock+0x177/0x190 [ 4444.091175] _raw_spin_lock+0x21/0x30 [ 4444.091180] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4444.091732] obd_mod_free+0x46/0x140 [ptlrpc] [ 4444.093209] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4444.093215] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4444.093221] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4444.093223] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4444.093226] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4444.093228] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4444.093230] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4444.093233] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4444.093236] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4444.093238] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4444.093244] Call Trace: [ 4444.093247] [ 4444.093250] ? show_regs.cold.9+0x22/0x2f [ 4444.093258] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4444.093407] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4444.093411] ? watchdog+0x40/0x40 [ 4444.093416] ? hrtimer_interrupt+0x128/0x2c0 [ 4444.093420] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4444.093426] ? apic_timer_interrupt+0xf/0x20 [ 4444.093431] [ 4444.093433] ? native_safe_halt+0x2e/0x50 [ 4444.093437] kvm_wait+0xcd/0xe0 [ 4444.093443] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4444.093449] do_raw_spin_lock+0x177/0x190 [ 4444.093453] _raw_spin_lock+0x21/0x30 [ 4444.093458] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4444.093760] ? do_raw_spin_unlock+0x75/0x190 [ 4444.093775] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4444.097983] mdc_commit_open+0x91/0xb0 [mdc] [ 4444.098029] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4444.098609] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4444.098831] after_reply+0xe2c/0x1780 [ptlrpc] [ 4444.099037] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4444.099247] ? __next_timer_interrupt+0x160/0x160 [ 4444.099258] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4444.099856] ? do_wait_intr+0xf0/0xf0 [ 4444.099867] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4444.100079] kthread+0x1d1/0x200 [ 4444.100085] ? set_kthread_struct+0x70/0x70 [ 4444.100089] ret_from_fork+0x1f/0x30 [ 4444.383341] ? do_wait_intr+0xf0/0xf0 [ 4444.385630] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4444.389520] mdc_reint+0x2d/0x110 [mdc] [ 4444.391586] mdc_unlink+0x658/0x870 [mdc] [ 4444.394280] lmv_unlink+0x34d/0xbd0 [lmv] [ 4444.396120] ll_unlink+0x24d/0x7f0 [lustre] [ 4444.398407] vfs_unlink+0x202/0x330 [ 4444.400830] do_unlinkat+0x2fa/0x460 [ 4444.405065] __x64_sys_unlink+0x37/0x50 [ 4444.407882] do_syscall_64+0xc1/0x440 [ 4444.410878] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4444.415653] RIP: 0033:0x7fe02c11910b [ 4444.417255] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4444.427653] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4444.432320] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4444.439334] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4444.445932] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4444.451485] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4444.455987] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4464.064022] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4464.069016] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4464.122390] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4464.136608] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4464.142758] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4464.146793] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4464.150543] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4464.161534] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4464.168295] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4464.172330] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4464.175209] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4464.179552] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4464.184017] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4464.186892] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4464.188945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4464.190657] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4464.193502] Call Trace: [ 4464.194686] [ 4464.195531] ? show_regs.cold.9+0x22/0x2f [ 4464.197119] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4464.199012] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4464.200750] ? watchdog+0x40/0x40 [ 4464.202071] ? hrtimer_interrupt+0x128/0x2c0 [ 4464.203671] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4464.205504] ? apic_timer_interrupt+0xf/0x20 [ 4464.207232] [ 4464.208042] ? native_safe_halt+0x2e/0x50 [ 4464.209553] kvm_wait+0xcd/0xe0 [ 4464.210762] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4464.212791] do_raw_spin_lock+0x177/0x190 [ 4464.214343] _raw_spin_lock+0x21/0x30 [ 4464.215723] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4464.217955] process_one_work+0x2c8/0x700 [ 4464.219479] worker_thread+0x296/0x6e0 [ 4464.220924] ? rescuer_thread+0x570/0x570 [ 4464.222592] kthread+0x1d1/0x200 [ 4464.225845] ? set_kthread_struct+0x70/0x70 [ 4464.229623] ret_from_fork+0x1f/0x30 [ 4472.060295] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4472.060328] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4472.068782] Modules linked in: [ 4472.073755] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 [ 4472.075677] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4472.075769] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4472.075775] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4472.075778] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4472.099483] squashfs [ 4472.112867] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4472.119319] ata_generic [ 4472.126104] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4472.126120] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4472.126122] RDX: 0000000000002106 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4472.126124] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4472.126126] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4472.126128] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4472.126131] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4472.126134] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4472.126136] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4472.126144] Call Trace: [ 4472.126151] [ 4472.126157] ? show_regs.cold.9+0x22/0x2f [ 4472.126170] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4472.126178] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4472.126182] ? watchdog+0x40/0x40 [ 4472.126186] ? hrtimer_interrupt+0x128/0x2c0 [ 4472.126190] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4472.126198] ? apic_timer_interrupt+0xf/0x20 [ 4472.126203] [ 4472.126206] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4472.126214] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4472.126219] do_raw_spin_lock+0x177/0x190 [ 4472.126223] _raw_spin_lock+0x21/0x30 [ 4472.126231] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4472.126670] obd_mod_free+0x46/0x140 [ptlrpc] [ 4472.134691] crct10dif_pclmul [ 4472.135290] mdc_commit_open+0x91/0xb0 [mdc] [ 4472.135337] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4472.135829] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4472.136037] after_reply+0xe2c/0x1780 [ptlrpc] [ 4472.136239] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4472.136462] ? __next_timer_interrupt+0x160/0x160 [ 4472.136475] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4472.136707] ? do_wait_intr+0xf0/0xf0 [ 4472.136717] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4472.156221] crc32_pclmul [ 4472.157190] kthread+0x1d1/0x200 [ 4472.163363] crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4472.163389] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4472.163397] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4472.163400] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4472.163416] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4472.163420] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4472.163425] RAX: 000000000000538d RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4472.163427] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4472.163429] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4472.163431] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4472.163433] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4472.163436] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4472.163438] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4472.163440] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4472.163449] Call Trace: [ 4472.163455] [ 4472.163461] ? show_regs.cold.9+0x22/0x2f [ 4472.163470] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4472.163477] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4472.163482] ? watchdog+0x40/0x40 [ 4472.163485] ? hrtimer_interrupt+0x128/0x2c0 [ 4472.163489] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4472.163497] ? apic_timer_interrupt+0xf/0x20 [ 4472.163503] [ 4472.163506] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4472.168345] ? set_kthread_struct+0x70/0x70 [ 4472.168363] ret_from_fork+0x1f/0x30 [ 4472.347404] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4472.349957] do_raw_spin_lock+0x177/0x190 [ 4472.351730] _raw_spin_lock+0x21/0x30 [ 4472.353521] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4472.356113] ? do_raw_spin_unlock+0x75/0x190 [ 4472.358199] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4472.361762] ? do_wait_intr+0xf0/0xf0 [ 4472.364374] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4472.368105] mdc_reint+0x2d/0x110 [mdc] [ 4472.369651] mdc_unlink+0x658/0x870 [mdc] [ 4472.371936] lmv_unlink+0x34d/0xbd0 [lmv] [ 4472.373922] ll_unlink+0x24d/0x7f0 [lustre] [ 4472.376294] vfs_unlink+0x202/0x330 [ 4472.378084] do_unlinkat+0x2fa/0x460 [ 4472.379717] __x64_sys_unlink+0x37/0x50 [ 4472.381408] do_syscall_64+0xc1/0x440 [ 4472.383394] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4472.387234] RIP: 0033:0x7fe02c11910b [ 4472.389029] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4472.397929] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4472.401836] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4472.404725] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4472.407086] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4472.409073] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4472.411805] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4492.064156] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4492.071487] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4492.132285] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4492.144829] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4492.150153] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4492.154841] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4492.159485] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4492.177267] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4492.187892] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4492.193743] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4492.199329] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4492.203671] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4492.208475] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4492.213472] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4492.218590] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4492.225063] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4492.229299] Call Trace: [ 4492.231741] [ 4492.233561] ? show_regs.cold.9+0x22/0x2f [ 4492.238662] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4492.244641] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4492.246587] ? watchdog+0x40/0x40 [ 4492.248272] ? hrtimer_interrupt+0x128/0x2c0 [ 4492.251727] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4492.253837] ? apic_timer_interrupt+0xf/0x20 [ 4492.262442] [ 4492.263566] ? native_safe_halt+0x2e/0x50 [ 4492.265597] kvm_wait+0xcd/0xe0 [ 4492.267472] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4492.271838] do_raw_spin_lock+0x177/0x190 [ 4492.274144] _raw_spin_lock+0x21/0x30 [ 4492.278738] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4492.284365] process_one_work+0x2c8/0x700 [ 4492.287677] worker_thread+0x296/0x6e0 [ 4492.290515] ? rescuer_thread+0x570/0x570 [ 4492.294199] kthread+0x1d1/0x200 [ 4492.296662] ? set_kthread_struct+0x70/0x70 [ 4492.301328] ret_from_fork+0x1f/0x30 [ 4500.059258] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4500.060061] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4500.070298] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4500.070375] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4500.070382] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4500.070385] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4500.074011] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4500.074092] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4500.074099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4500.074102] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4500.074117] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4500.074121] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4500.074126] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4500.074128] RDX: 0000000000007f81 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4500.074130] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4500.074282] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4500.074285] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4500.074288] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4500.074291] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4500.074293] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4500.074302] Call Trace: [ 4500.074308] [ 4500.074315] ? show_regs.cold.9+0x22/0x2f [ 4500.074325] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4500.074332] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4500.074336] ? watchdog+0x40/0x40 [ 4500.074340] ? hrtimer_interrupt+0x128/0x2c0 [ 4500.074344] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4500.074355] ? apic_timer_interrupt+0xf/0x20 [ 4500.074359] [ 4500.074361] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4500.074366] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4500.074371] do_raw_spin_lock+0x177/0x190 [ 4500.074376] _raw_spin_lock+0x21/0x30 [ 4500.074381] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4500.074945] obd_mod_free+0x46/0x140 [ptlrpc] [ 4500.133498] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4500.138236] mdc_commit_open+0x91/0xb0 [mdc] [ 4500.138291] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4500.138570] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4500.138761] after_reply+0xe2c/0x1780 [ptlrpc] [ 4500.138963] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4500.139150] ? __next_timer_interrupt+0x160/0x160 [ 4500.139161] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4500.139433] ? do_wait_intr+0xf0/0xf0 [ 4500.139442] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4500.139640] kthread+0x1d1/0x200 [ 4500.139646] ? set_kthread_struct+0x70/0x70 [ 4500.139649] ret_from_fork+0x1f/0x30 [ 4500.371077] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4500.373669] RAX: 00000000000078b5 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4500.376378] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4500.378921] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4500.381365] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4500.383770] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4500.386746] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4500.389318] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4500.391112] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4500.392960] Call Trace: [ 4500.393629] [ 4500.394086] ? show_regs.cold.9+0x22/0x2f [ 4500.395089] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4500.396386] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4500.398070] ? watchdog+0x40/0x40 [ 4500.398975] ? hrtimer_interrupt+0x128/0x2c0 [ 4500.400060] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4500.401497] ? apic_timer_interrupt+0xf/0x20 [ 4500.402594] [ 4500.403070] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4500.404912] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4500.406467] do_raw_spin_lock+0x177/0x190 [ 4500.407606] _raw_spin_lock+0x21/0x30 [ 4500.408409] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4500.409774] ? do_raw_spin_unlock+0x75/0x190 [ 4500.410754] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4500.412720] ? do_wait_intr+0xf0/0xf0 [ 4500.413624] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4500.415108] mdc_reint+0x2d/0x110 [mdc] [ 4500.416414] mdc_unlink+0x658/0x870 [mdc] [ 4500.417605] lmv_unlink+0x34d/0xbd0 [lmv] [ 4500.418615] ll_unlink+0x24d/0x7f0 [lustre] [ 4500.419850] vfs_unlink+0x202/0x330 [ 4500.420732] do_unlinkat+0x2fa/0x460 [ 4500.421640] __x64_sys_unlink+0x37/0x50 [ 4500.422618] do_syscall_64+0xc1/0x440 [ 4500.423572] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4500.424833] RIP: 0033:0x7fe02c11910b [ 4500.425760] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4500.433431] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4500.436646] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4500.439603] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4500.442645] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4500.445737] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4500.448701] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4520.065253] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4520.070124] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4520.095942] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4520.103475] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4520.108711] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4520.112881] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4520.116419] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4520.129984] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4520.134675] RAX: 0000000000006fec RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4520.138971] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4520.144062] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4520.149032] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4520.152314] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4520.156953] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4520.162855] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4520.166463] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4520.170814] Call Trace: [ 4520.172764] [ 4520.173811] ? show_regs.cold.9+0x22/0x2f [ 4520.176301] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4520.178986] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4520.180561] ? watchdog+0x40/0x40 [ 4520.182088] ? hrtimer_interrupt+0x128/0x2c0 [ 4520.184900] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4520.187846] ? apic_timer_interrupt+0xf/0x20 [ 4520.190414] [ 4520.191962] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4520.196040] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4520.200165] do_raw_spin_lock+0x177/0x190 [ 4520.202275] _raw_spin_lock+0x21/0x30 [ 4520.204344] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4520.207932] process_one_work+0x2c8/0x700 [ 4520.209743] worker_thread+0x296/0x6e0 [ 4520.211762] ? rescuer_thread+0x570/0x570 [ 4520.214149] kthread+0x1d1/0x200 [ 4520.216068] ? set_kthread_struct+0x70/0x70 [ 4520.218281] ret_from_fork+0x1f/0x30 [ 4528.057176] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4528.060736] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4528.061145] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4528.080628] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4528.080653] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4528.080656] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4528.080675] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4528.080679] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4528.080685] RAX: 0000000000004539 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4528.080688] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4528.080690] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4528.080691] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4528.080694] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4528.080700] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4528.080703] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4528.080705] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4528.080712] Call Trace: [ 4528.080718] [ 4528.080723] ? show_regs.cold.9+0x22/0x2f [ 4528.080732] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4528.080737] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4528.080741] ? watchdog+0x40/0x40 [ 4528.080745] ? hrtimer_interrupt+0x128/0x2c0 [ 4528.080748] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4528.080756] ? apic_timer_interrupt+0xf/0x20 [ 4528.080760] [ 4528.080763] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4528.080767] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4528.080772] do_raw_spin_lock+0x177/0x190 [ 4528.080777] _raw_spin_lock+0x21/0x30 [ 4528.080783] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4528.081196] ? do_raw_spin_unlock+0x75/0x190 [ 4528.081202] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4528.081425] ? do_wait_intr+0xf0/0xf0 [ 4528.081433] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4528.081852] mdc_reint+0x2d/0x110 [mdc] [ 4528.081908] mdc_unlink+0x658/0x870 [mdc] [ 4528.081942] lmv_unlink+0x34d/0xbd0 [lmv] [ 4528.081979] ll_unlink+0x24d/0x7f0 [lustre] [ 4528.082171] vfs_unlink+0x202/0x330 [ 4528.082178] do_unlinkat+0x2fa/0x460 [ 4528.082188] __x64_sys_unlink+0x37/0x50 [ 4528.082191] do_syscall_64+0xc1/0x440 [ 4528.082201] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4528.082210] RIP: 0033:0x7fe02c11910b [ 4528.082217] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4528.082225] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4528.082230] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4528.082232] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4528.082234] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4528.082241] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4528.082243] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4528.219467] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4528.243944] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4528.250094] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4528.254241] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4528.256939] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4528.266968] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4528.270936] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4528.274695] RDX: 0000000000007e45 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4528.278056] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4528.281303] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4528.284799] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4528.288146] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4528.291614] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4528.294443] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4528.297790] Call Trace: [ 4528.298863] [ 4528.299741] ? show_regs.cold.9+0x22/0x2f [ 4528.301497] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4528.303634] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4528.305812] ? watchdog+0x40/0x40 [ 4528.307169] ? hrtimer_interrupt+0x128/0x2c0 [ 4528.308612] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4528.310756] ? apic_timer_interrupt+0xf/0x20 [ 4528.312518] [ 4528.313543] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4528.315926] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4528.317960] do_raw_spin_lock+0x177/0x190 [ 4528.319901] _raw_spin_lock+0x21/0x30 [ 4528.321276] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4528.324346] obd_mod_free+0x46/0x140 [ptlrpc] [ 4528.327772] mdc_commit_open+0x91/0xb0 [mdc] [ 4528.329211] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4528.331815] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4528.334513] after_reply+0xe2c/0x1780 [ptlrpc] [ 4528.336961] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4528.339383] ? __next_timer_interrupt+0x160/0x160 [ 4528.341806] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4528.344014] ? do_wait_intr+0xf0/0xf0 [ 4528.345829] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4528.348328] kthread+0x1d1/0x200 [ 4528.350087] ? set_kthread_struct+0x70/0x70 [ 4528.352074] ret_from_fork+0x1f/0x30 [ 4542.146278] rcu: INFO: rcu_sched self-detected stall on CPU [ 4542.148968] rcu: 1-....: (1634355 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=465735 [ 4542.159127] (t=2400054 jiffies g=278549 q=13081) [ 4542.166433] Sending NMI from CPU 1 to CPUs 0: [ 4542.171215] NMI backtrace for cpu 0 [ 4542.171221] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4542.171223] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4542.171227] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4542.171229] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4542.171232] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 [ 4542.171234] RAX: 0000000000007f8a RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4542.171236] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4542.171237] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4542.171238] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4542.171239] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4542.171241] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4542.171243] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4542.171244] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4542.171245] Call Trace: [ 4542.171246] [ 4542.171247] ? show_regs.cold.9+0x22/0x2f [ 4542.171248] ? nmi_cpu_backtrace.cold.8+0x98/0xa2 [ 4542.171250] ? nmi_cpu_backtrace_handler+0x15/0x40 [ 4542.171251] ? nmi_handle+0xbd/0x210 [ 4542.171252] ? default_do_nmi+0x4b/0x1b0 [ 4542.171253] ? do_nmi+0x280/0x380 [ 4542.171254] ? end_repeat_nmi+0x16/0x1f [ 4542.171255] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4542.171256] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4542.171257] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4542.171258] [ 4542.171259] do_raw_spin_lock+0x177/0x190 [ 4542.171260] _raw_spin_lock+0x21/0x30 [ 4542.171261] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4542.171262] ? do_raw_spin_unlock+0x75/0x190 [ 4542.171263] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4542.171264] ? do_wait_intr+0xf0/0xf0 [ 4542.171265] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4542.171266] mdc_reint+0x2d/0x110 [mdc] [ 4542.171267] mdc_unlink+0x658/0x870 [mdc] [ 4542.171268] lmv_unlink+0x34d/0xbd0 [lmv] [ 4542.171268] ll_unlink+0x24d/0x7f0 [lustre] [ 4542.171269] vfs_unlink+0x202/0x330 [ 4542.171270] do_unlinkat+0x2fa/0x460 [ 4542.171271] __x64_sys_unlink+0x37/0x50 [ 4542.171272] do_syscall_64+0xc1/0x440 [ 4542.171272] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4542.171273] RIP: 0033:0x7fe02c11910b [ 4542.171275] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4542.171277] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4542.171279] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4542.171283] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4542.171284] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4542.171285] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4542.171287] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4542.171293] NMI backtrace for cpu 1 [ 4542.331905] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4542.336927] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4542.340958] Call Trace: [ 4542.341943] [ 4542.342830] ? dump_stack+0xbb/0x10e [ 4542.344121] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4542.346152] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4542.348490] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4542.350944] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4542.352737] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4542.354783] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4542.355947] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4542.358376] ? update_process_times+0x6d/0xc0 [ 4542.363216] ? tick_sched_handle.isra.18+0x27/0x90 [ 4542.366240] ? tick_sched_timer+0x53/0xb0 [ 4542.368854] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4542.371039] ? tick_sched_handle.isra.18+0x90/0x90 [ 4542.378793] ? hrtimer_interrupt+0x128/0x2c0 [ 4542.380037] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4542.386107] ? apic_timer_interrupt+0xf/0x20 [ 4542.388598] [ 4542.389962] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4542.392750] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4542.402067] ? do_raw_spin_lock+0x177/0x190 [ 4542.407922] ? _raw_spin_lock+0x21/0x30 [ 4542.410671] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4542.413992] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4542.417032] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4542.419133] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4542.421580] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4542.424179] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4542.426319] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4542.429614] ? __next_timer_interrupt+0x160/0x160 [ 4542.430994] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4542.434624] ? do_wait_intr+0xf0/0xf0 [ 4542.435954] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4542.440611] ? kthread+0x1d1/0x200 [ 4542.443214] ? set_kthread_struct+0x70/0x70 [ 4542.445701] ? ret_from_fork+0x1f/0x30 [ 4548.065277] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4548.069899] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4548.096466] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4548.105048] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4548.112028] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4548.114752] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4548.116698] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4548.126230] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4548.129547] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4548.132502] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4548.136138] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4548.142039] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4548.147984] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4548.154644] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4548.158246] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4548.163559] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4548.168227] Call Trace: [ 4548.171800] [ 4548.172365] ? show_regs.cold.9+0x22/0x2f [ 4548.173612] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4548.175780] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4548.177121] ? watchdog+0x40/0x40 [ 4548.178154] ? hrtimer_interrupt+0x128/0x2c0 [ 4548.179507] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4548.181006] ? apic_timer_interrupt+0xf/0x20 [ 4548.182883] [ 4548.183625] ? native_safe_halt+0x2e/0x50 [ 4548.185846] kvm_wait+0xcd/0xe0 [ 4548.187109] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4548.189070] do_raw_spin_lock+0x177/0x190 [ 4548.190743] _raw_spin_lock+0x21/0x30 [ 4548.192811] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4548.195010] process_one_work+0x2c8/0x700 [ 4548.197781] worker_thread+0x296/0x6e0 [ 4548.199775] ? rescuer_thread+0x570/0x570 [ 4548.201555] kthread+0x1d1/0x200 [ 4548.202757] ? set_kthread_struct+0x70/0x70 [ 4548.205025] ret_from_fork+0x1f/0x30 [ 4568.057026] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4568.060027] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4568.060304] Modules linked in: [ 4568.064908] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4568.064985] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4568.064993] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4568.064995] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4568.065009] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4568.065014] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4568.065019] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4568.065021] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4568.065023] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4568.065025] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4568.065027] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4568.065030] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4568.065033] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4568.065036] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4568.065045] Call Trace: [ 4568.065050] [ 4568.065056] ? show_regs.cold.9+0x22/0x2f [ 4568.065066] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4568.065075] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4568.065080] ? watchdog+0x40/0x40 [ 4568.065085] ? hrtimer_interrupt+0x128/0x2c0 [ 4568.065090] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4568.065099] ? apic_timer_interrupt+0xf/0x20 [ 4568.065105] [ 4568.065108] ? native_safe_halt+0x2e/0x50 [ 4568.065112] kvm_wait+0xcd/0xe0 [ 4568.065120] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4568.065128] do_raw_spin_lock+0x177/0x190 [ 4568.065132] _raw_spin_lock+0x21/0x30 [ 4568.065137] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4568.066676] obd_mod_free+0x46/0x140 [ptlrpc] [ 4568.067606] lustre(O) [ 4568.098964] mdc_commit_open+0x91/0xb0 [mdc] [ 4568.099085] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4568.109860] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4568.109933] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4568.109941] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4568.109944] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4568.109956] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4568.109960] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4568.109964] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4568.109967] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4568.109969] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4568.109971] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4568.109973] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4568.109977] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4568.110186] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4568.110188] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4568.110196] Call Trace: [ 4568.110202] [ 4568.110208] ? show_regs.cold.9+0x22/0x2f [ 4568.110218] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4568.110224] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4568.110229] ? watchdog+0x40/0x40 [ 4568.110233] ? hrtimer_interrupt+0x128/0x2c0 [ 4568.110237] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4568.110244] ? apic_timer_interrupt+0xf/0x20 [ 4568.110249] [ 4568.110251] ? native_safe_halt+0x2e/0x50 [ 4568.110255] kvm_wait+0xcd/0xe0 [ 4568.110262] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4568.110269] do_raw_spin_lock+0x177/0x190 [ 4568.110274] _raw_spin_lock+0x21/0x30 [ 4568.110279] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4568.120716] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4568.127451] ? do_raw_spin_unlock+0x75/0x190 [ 4568.140115] after_reply+0xe2c/0x1780 [ptlrpc] [ 4568.140460] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4568.141049] ? __next_timer_interrupt+0x160/0x160 [ 4568.141060] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4568.141263] ? do_wait_intr+0xf0/0xf0 [ 4568.141272] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4568.141471] kthread+0x1d1/0x200 [ 4568.141477] ? set_kthread_struct+0x70/0x70 [ 4568.141482] ret_from_fork+0x1f/0x30 [ 4568.460289] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4568.465351] ? do_wait_intr+0xf0/0xf0 [ 4568.467508] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4568.472784] mdc_reint+0x2d/0x110 [mdc] [ 4568.476929] mdc_unlink+0x658/0x870 [mdc] [ 4568.482159] lmv_unlink+0x34d/0xbd0 [lmv] [ 4568.483741] ll_unlink+0x24d/0x7f0 [lustre] [ 4568.489901] vfs_unlink+0x202/0x330 [ 4568.492863] do_unlinkat+0x2fa/0x460 [ 4568.494894] __x64_sys_unlink+0x37/0x50 [ 4568.497958] do_syscall_64+0xc1/0x440 [ 4568.499394] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4568.502547] RIP: 0033:0x7fe02c11910b [ 4568.504560] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4568.516252] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4568.519721] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4568.523312] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4568.529845] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4568.534132] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4568.536860] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4576.067731] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4576.077011] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4576.111013] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4576.115248] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4576.119283] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4576.123293] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4576.126287] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4576.134232] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4576.137629] RAX: 0000000000002d92 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4576.141192] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4576.144651] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4576.148202] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4576.152425] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4576.156485] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4576.160199] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4576.166058] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4576.171296] Call Trace: [ 4576.172477] [ 4576.173394] ? show_regs.cold.9+0x22/0x2f [ 4576.175227] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4576.177576] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4576.179243] ? watchdog+0x40/0x40 [ 4576.182743] ? hrtimer_interrupt+0x128/0x2c0 [ 4576.184646] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4576.186410] ? apic_timer_interrupt+0xf/0x20 [ 4576.189103] [ 4576.189764] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4576.194129] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4576.196746] do_raw_spin_lock+0x177/0x190 [ 4576.198452] _raw_spin_lock+0x21/0x30 [ 4576.200124] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4576.203630] process_one_work+0x2c8/0x700 [ 4576.205510] worker_thread+0x296/0x6e0 [ 4576.208766] ? rescuer_thread+0x570/0x570 [ 4576.212663] kthread+0x1d1/0x200 [ 4576.214339] ? set_kthread_struct+0x70/0x70 [ 4576.216360] ret_from_fork+0x1f/0x30 [ 4596.057152] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4596.060340] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4596.061850] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 [ 4596.066260] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4596.066341] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4596.066348] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4596.066350] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4596.066406] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4596.066411] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4596.066416] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4596.066419] RDX: 0000000000007e4b RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4596.066421] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4596.066423] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4596.066425] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4596.066428] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4596.066431] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4596.066433] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4596.066441] Call Trace: [ 4596.066457] [ 4596.066463] ? show_regs.cold.9+0x22/0x2f [ 4596.066471] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4596.066478] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4596.066482] ? watchdog+0x40/0x40 [ 4596.066486] ? hrtimer_interrupt+0x128/0x2c0 [ 4596.066590] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4596.066598] ? apic_timer_interrupt+0xf/0x20 [ 4596.066602] [ 4596.066605] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4596.066609] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4596.066613] do_raw_spin_lock+0x177/0x190 [ 4596.066618] _raw_spin_lock+0x21/0x30 [ 4596.066623] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4596.086331] squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4596.086436] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4596.086547] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4596.086550] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4596.086564] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4596.086568] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4596.086573] RAX: 0000000000005aee RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4596.122415] obd_mod_free+0x46/0x140 [ptlrpc] [ 4596.129960] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4596.129969] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4596.129972] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4596.129974] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4596.129978] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4596.129981] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4596.129983] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4596.129991] Call Trace: [ 4596.129998] [ 4596.130004] ? show_regs.cold.9+0x22/0x2f [ 4596.130018] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4596.130025] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4596.130030] ? watchdog+0x40/0x40 [ 4596.130034] ? hrtimer_interrupt+0x128/0x2c0 [ 4596.130038] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4596.130048] ? apic_timer_interrupt+0xf/0x20 [ 4596.130053] [ 4596.130056] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4596.130062] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4596.130066] do_raw_spin_lock+0x177/0x190 [ 4596.130071] _raw_spin_lock+0x21/0x30 [ 4596.130078] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4596.130361] ? do_raw_spin_unlock+0x75/0x190 [ 4596.130367] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4596.131018] ? do_wait_intr+0xf0/0xf0 [ 4596.131027] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4596.137094] mdc_commit_open+0x91/0xb0 [mdc] [ 4596.141967] mdc_reint+0x2d/0x110 [mdc] [ 4596.142017] mdc_unlink+0x658/0x870 [mdc] [ 4596.142045] lmv_unlink+0x34d/0xbd0 [lmv] [ 4596.142079] ll_unlink+0x24d/0x7f0 [lustre] [ 4596.142207] vfs_unlink+0x202/0x330 [ 4596.142215] do_unlinkat+0x2fa/0x460 [ 4596.142221] __x64_sys_unlink+0x37/0x50 [ 4596.142224] do_syscall_64+0xc1/0x440 [ 4596.142231] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4596.142240] RIP: 0033:0x7fe02c11910b [ 4596.142247] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4596.142251] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4596.142256] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4596.142258] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4596.142260] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4596.142263] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4596.142265] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4596.575301] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4596.580995] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4596.589698] after_reply+0xe2c/0x1780 [ptlrpc] [ 4596.594452] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4596.597414] ? __next_timer_interrupt+0x160/0x160 [ 4596.603196] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4596.605987] ? do_wait_intr+0xf0/0xf0 [ 4596.607774] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4596.610286] kthread+0x1d1/0x200 [ 4596.611601] ? set_kthread_struct+0x70/0x70 [ 4596.613402] ret_from_fork+0x1f/0x30 [ 4604.064024] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4604.068068] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4604.096499] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4604.103499] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4604.111156] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4604.120941] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4604.124874] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4604.145470] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4604.157510] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4604.166317] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4604.172125] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4604.176937] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4604.181049] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4604.192475] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4604.199496] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4604.202110] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4604.205848] Call Trace: [ 4604.206885] [ 4604.207879] ? show_regs.cold.9+0x22/0x2f [ 4604.213712] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4604.216587] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4604.218597] ? watchdog+0x40/0x40 [ 4604.220857] ? hrtimer_interrupt+0x128/0x2c0 [ 4604.222742] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4604.224322] ? apic_timer_interrupt+0xf/0x20 [ 4604.225708] [ 4604.226481] ? native_safe_halt+0x2e/0x50 [ 4604.228251] kvm_wait+0xcd/0xe0 [ 4604.229643] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4604.231815] do_raw_spin_lock+0x177/0x190 [ 4604.233729] _raw_spin_lock+0x21/0x30 [ 4604.235454] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4604.238726] process_one_work+0x2c8/0x700 [ 4604.240853] worker_thread+0x296/0x6e0 [ 4604.242748] ? rescuer_thread+0x570/0x570 [ 4604.244104] kthread+0x1d1/0x200 [ 4604.245470] ? set_kthread_struct+0x70/0x70 [ 4604.246876] ret_from_fork+0x1f/0x30 [ 4624.058600] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4624.060487] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4624.061064] Modules linked in: [ 4624.070811] Modules linked in: [ 4624.070813] lustre(O) [ 4624.074897] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4624.074982] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4624.074990] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4624.074993] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4624.076491] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4624.076562] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4624.076569] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4624.076571] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4624.076583] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4624.076588] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4624.076593] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4624.076595] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4624.076597] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4624.076600] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4624.076602] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4624.076605] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4624.076609] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4624.076611] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4624.076626] Call Trace: [ 4624.076632] [ 4624.076637] ? show_regs.cold.9+0x22/0x2f [ 4624.076647] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4624.076654] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4624.076659] ? watchdog+0x40/0x40 [ 4624.076663] ? hrtimer_interrupt+0x128/0x2c0 [ 4624.076667] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4624.076674] ? apic_timer_interrupt+0xf/0x20 [ 4624.076679] [ 4624.076899] ? native_safe_halt+0x2e/0x50 [ 4624.076906] kvm_wait+0xcd/0xe0 [ 4624.076915] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4624.076922] do_raw_spin_lock+0x177/0x190 [ 4624.076926] _raw_spin_lock+0x21/0x30 [ 4624.076932] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4624.077233] ? do_raw_spin_unlock+0x75/0x190 [ 4624.077239] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4624.080414] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4624.080430] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4624.080437] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4624.080439] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4624.080441] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4624.080444] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4624.080446] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4624.080449] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4624.080452] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4624.080454] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4624.080462] Call Trace: [ 4624.080468] [ 4624.080475] ? show_regs.cold.9+0x22/0x2f [ 4624.080492] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4624.080499] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4624.080504] ? watchdog+0x40/0x40 [ 4624.080508] ? hrtimer_interrupt+0x128/0x2c0 [ 4624.080513] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4624.080524] ? apic_timer_interrupt+0xf/0x20 [ 4624.080528] [ 4624.080531] ? native_safe_halt+0x2e/0x50 [ 4624.080535] kvm_wait+0xcd/0xe0 [ 4624.080543] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4624.080551] do_raw_spin_lock+0x177/0x190 [ 4624.080555] _raw_spin_lock+0x21/0x30 [ 4624.080560] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4624.108972] ? do_wait_intr+0xf0/0xf0 [ 4624.108999] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4624.115389] obd_mod_free+0x46/0x140 [ptlrpc] [ 4624.119008] mdc_reint+0x2d/0x110 [mdc] [ 4624.121045] mdc_commit_open+0x91/0xb0 [mdc] [ 4624.138674] mdc_unlink+0x658/0x870 [mdc] [ 4624.151394] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4624.154634] lmv_unlink+0x34d/0xbd0 [lmv] [ 4624.155078] ll_unlink+0x24d/0x7f0 [lustre] [ 4624.155201] vfs_unlink+0x202/0x330 [ 4624.155211] do_unlinkat+0x2fa/0x460 [ 4624.155217] __x64_sys_unlink+0x37/0x50 [ 4624.155220] do_syscall_64+0xc1/0x440 [ 4624.155228] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4624.155237] RIP: 0033:0x7fe02c11910b [ 4624.155245] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4624.155249] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4624.155255] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4624.155258] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4624.155260] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4624.155263] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4624.155265] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4624.545498] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4624.549962] after_reply+0xe2c/0x1780 [ptlrpc] [ 4624.554491] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4624.558661] ? __next_timer_interrupt+0x160/0x160 [ 4624.562428] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4624.566032] ? do_wait_intr+0xf0/0xf0 [ 4624.568506] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4624.573532] kthread+0x1d1/0x200 [ 4624.574485] ? set_kthread_struct+0x70/0x70 [ 4624.576163] ret_from_fork+0x1f/0x30 [ 4632.065233] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4632.071233] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4632.096708] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4632.102273] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4632.106109] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4632.109752] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4632.112997] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4632.123388] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4632.130264] RAX: 00000000000073a5 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4632.135651] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4632.143813] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4632.151127] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4632.156137] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4632.163919] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4632.170102] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4632.172228] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4632.178145] Call Trace: [ 4632.181541] [ 4632.183703] ? show_regs.cold.9+0x22/0x2f [ 4632.186058] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4632.192641] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4632.194185] ? watchdog+0x40/0x40 [ 4632.199362] ? hrtimer_interrupt+0x128/0x2c0 [ 4632.201283] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4632.204837] ? apic_timer_interrupt+0xf/0x20 [ 4632.206950] [ 4632.208111] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4632.213093] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4632.215089] do_raw_spin_lock+0x177/0x190 [ 4632.226304] _raw_spin_lock+0x21/0x30 [ 4632.229360] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4632.232288] process_one_work+0x2c8/0x700 [ 4632.234304] worker_thread+0x296/0x6e0 [ 4632.240740] ? rescuer_thread+0x570/0x570 [ 4632.243468] kthread+0x1d1/0x200 [ 4632.249782] ? set_kthread_struct+0x70/0x70 [ 4632.251243] ret_from_fork+0x1f/0x30 [ 4652.057025] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4652.060027] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4652.061399] Modules linked in: lustre(O) [ 4652.064066] Modules linked in: [ 4652.064069] osp(O) ofd(O) [ 4652.065193] lustre(O) [ 4652.067064] lod(O) [ 4652.067942] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4652.068011] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4652.068017] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4652.068020] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4652.068032] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4652.068037] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4652.068041] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4652.068044] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4652.068046] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4652.068048] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4652.068050] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4652.068054] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4652.068056] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4652.068058] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4652.068065] Call Trace: [ 4652.068069] [ 4652.068075] ? show_regs.cold.9+0x22/0x2f [ 4652.068083] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4652.068089] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4652.069673] mdt(O) [ 4652.070363] ? watchdog+0x40/0x40 [ 4652.093882] mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4652.093951] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4652.093957] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4652.093960] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4652.093973] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4652.093977] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4652.093982] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4652.093984] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4652.093986] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4652.093988] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4652.093990] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4652.093994] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4652.093996] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4652.093998] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4652.094005] Call Trace: [ 4652.094010] [ 4652.094015] ? show_regs.cold.9+0x22/0x2f [ 4652.094024] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4652.094030] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4652.094034] ? watchdog+0x40/0x40 [ 4652.094038] ? hrtimer_interrupt+0x128/0x2c0 [ 4652.094042] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4652.094048] ? apic_timer_interrupt+0xf/0x20 [ 4652.094052] [ 4652.094056] ? native_safe_halt+0x2e/0x50 [ 4652.094060] kvm_wait+0xcd/0xe0 [ 4652.094067] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4652.094073] do_raw_spin_lock+0x177/0x190 [ 4652.094077] _raw_spin_lock+0x21/0x30 [ 4652.094081] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4652.094390] ? do_raw_spin_unlock+0x75/0x190 [ 4652.094396] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4652.102974] ? hrtimer_interrupt+0x128/0x2c0 [ 4652.102991] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4652.103000] ? apic_timer_interrupt+0xf/0x20 [ 4652.103005] [ 4652.103009] ? native_safe_halt+0x2e/0x50 [ 4652.103017] kvm_wait+0xcd/0xe0 [ 4652.103026] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4652.103034] do_raw_spin_lock+0x177/0x190 [ 4652.103038] _raw_spin_lock+0x21/0x30 [ 4652.103047] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4652.103358] obd_mod_free+0x46/0x140 [ptlrpc] [ 4652.103576] mdc_commit_open+0x91/0xb0 [mdc] [ 4652.103626] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4652.103964] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4652.104185] after_reply+0xe2c/0x1780 [ptlrpc] [ 4652.104393] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4652.104663] ? __next_timer_interrupt+0x160/0x160 [ 4652.104672] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4652.104954] ? do_wait_intr+0xf0/0xf0 [ 4652.104963] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4652.105190] kthread+0x1d1/0x200 [ 4652.105195] ? set_kthread_struct+0x70/0x70 [ 4652.105199] ret_from_fork+0x1f/0x30 [ 4652.232752] ? do_wait_intr+0xf0/0xf0 [ 4652.233704] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4652.235276] mdc_reint+0x2d/0x110 [mdc] [ 4652.236602] mdc_unlink+0x658/0x870 [mdc] [ 4652.237908] lmv_unlink+0x34d/0xbd0 [lmv] [ 4652.238987] ll_unlink+0x24d/0x7f0 [lustre] [ 4652.240332] vfs_unlink+0x202/0x330 [ 4652.241264] do_unlinkat+0x2fa/0x460 [ 4652.242228] __x64_sys_unlink+0x37/0x50 [ 4652.243205] do_syscall_64+0xc1/0x440 [ 4652.244214] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4652.245825] RIP: 0033:0x7fe02c11910b [ 4652.247180] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4652.254715] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4652.257207] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4652.259542] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4652.261652] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4652.263710] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4652.265574] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4660.064028] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4660.073280] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4660.116113] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4660.126626] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4660.133146] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4660.138277] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4660.142646] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4660.166449] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4660.172309] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4660.178761] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4660.184721] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4660.194433] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4660.203802] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4660.215236] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4660.232078] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4660.239088] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4660.245817] Call Trace: [ 4660.251468] [ 4660.252341] ? show_regs.cold.9+0x22/0x2f [ 4660.256162] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4660.257847] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4660.263291] ? watchdog+0x40/0x40 [ 4660.265737] ? hrtimer_interrupt+0x128/0x2c0 [ 4660.268058] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4660.272913] ? apic_timer_interrupt+0xf/0x20 [ 4660.277065] [ 4660.278135] ? native_safe_halt+0x2e/0x50 [ 4660.281136] kvm_wait+0xcd/0xe0 [ 4660.282991] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4660.285301] do_raw_spin_lock+0x177/0x190 [ 4660.291308] _raw_spin_lock+0x21/0x30 [ 4660.294223] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4660.297538] process_one_work+0x2c8/0x700 [ 4660.300167] worker_thread+0x296/0x6e0 [ 4660.301843] ? rescuer_thread+0x570/0x570 [ 4660.303916] kthread+0x1d1/0x200 [ 4660.306334] ? set_kthread_struct+0x70/0x70 [ 4660.314025] ret_from_fork+0x1f/0x30 [ 4680.059201] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4680.064801] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4680.068634] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4680.070841] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4680.070850] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4680.070852] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4680.070866] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4680.070870] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4680.070874] RAX: 0000000000002705 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4680.070877] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4680.070878] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4680.070880] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4680.070883] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4680.070886] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4680.070888] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4680.070890] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4680.070900] Call Trace: [ 4680.070905] [ 4680.070911] ? show_regs.cold.9+0x22/0x2f [ 4680.070921] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4680.070926] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4680.070930] ? watchdog+0x40/0x40 [ 4680.070935] ? hrtimer_interrupt+0x128/0x2c0 [ 4680.070939] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4680.070948] ? apic_timer_interrupt+0xf/0x20 [ 4680.070952] [ 4680.070955] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4680.070960] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4680.070984] do_raw_spin_lock+0x177/0x190 [ 4680.070990] _raw_spin_lock+0x21/0x30 [ 4680.070996] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4680.073999] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4680.074081] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4680.074088] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4680.074091] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4680.074105] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4680.074109] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4680.074114] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4680.074116] RDX: 0000000000000fae RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4680.074118] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4680.074120] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4680.074123] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4680.074126] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4680.074129] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4680.074131] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4680.074139] Call Trace: [ 4680.074144] [ 4680.074149] ? show_regs.cold.9+0x22/0x2f [ 4680.074158] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4680.074164] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4680.074168] ? watchdog+0x40/0x40 [ 4680.074173] ? hrtimer_interrupt+0x128/0x2c0 [ 4680.074176] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4680.074959] ? apic_timer_interrupt+0xf/0x20 [ 4680.074964] [ 4680.074967] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4680.074973] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4680.074978] do_raw_spin_lock+0x177/0x190 [ 4680.074983] _raw_spin_lock+0x21/0x30 [ 4680.074988] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4680.075345] obd_mod_free+0x46/0x140 [ptlrpc] [ 4680.075549] mdc_commit_open+0x91/0xb0 [mdc] [ 4680.075588] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4680.076629] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4680.076857] after_reply+0xe2c/0x1780 [ptlrpc] [ 4680.077926] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4680.078135] ? __next_timer_interrupt+0x160/0x160 [ 4680.078145] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4680.141612] ? do_raw_spin_unlock+0x75/0x190 [ 4680.141633] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4680.154263] ? do_wait_intr+0xf0/0xf0 [ 4680.154285] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4680.154579] kthread+0x1d1/0x200 [ 4680.154585] ? set_kthread_struct+0x70/0x70 [ 4680.154588] ret_from_fork+0x1f/0x30 [ 4680.703551] ? do_wait_intr+0xf0/0xf0 [ 4680.710838] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4680.720795] mdc_reint+0x2d/0x110 [mdc] [ 4680.723164] mdc_unlink+0x658/0x870 [mdc] [ 4680.724572] lmv_unlink+0x34d/0xbd0 [lmv] [ 4680.729793] ll_unlink+0x24d/0x7f0 [lustre] [ 4680.735120] vfs_unlink+0x202/0x330 [ 4680.738988] do_unlinkat+0x2fa/0x460 [ 4680.755149] __x64_sys_unlink+0x37/0x50 [ 4680.760917] do_syscall_64+0xc1/0x440 [ 4680.763711] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4680.778862] RIP: 0033:0x7fe02c11910b [ 4680.786147] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4680.811082] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4680.818874] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4680.830738] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4680.846184] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4680.857598] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4680.862510] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4688.064963] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4688.071015] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4688.093429] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4688.099403] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4688.103684] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4688.107672] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4688.111172] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4688.119440] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4688.123560] RAX: 000000000000738f RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4688.127061] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4688.131043] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4688.134419] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4688.137250] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4688.140933] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4688.144464] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4688.147557] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4688.151667] Call Trace: [ 4688.153419] [ 4688.154725] ? show_regs.cold.9+0x22/0x2f [ 4688.157362] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4688.159985] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4688.162059] ? watchdog+0x40/0x40 [ 4688.164072] ? hrtimer_interrupt+0x128/0x2c0 [ 4688.166063] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4688.168667] ? apic_timer_interrupt+0xf/0x20 [ 4688.171373] [ 4688.172452] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4688.175448] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4688.178378] do_raw_spin_lock+0x177/0x190 [ 4688.180448] _raw_spin_lock+0x21/0x30 [ 4688.182584] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4688.186109] process_one_work+0x2c8/0x700 [ 4688.188468] worker_thread+0x296/0x6e0 [ 4688.190605] ? rescuer_thread+0x570/0x570 [ 4688.192428] kthread+0x1d1/0x200 [ 4688.194489] ? set_kthread_struct+0x70/0x70 [ 4688.196789] ret_from_fork+0x1f/0x30 [ 4708.057008] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4708.064666] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4708.071524] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4708.071616] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4708.071623] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4708.071626] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4708.071676] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4708.071681] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4708.071686] RAX: 0000000000001b25 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4708.071688] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4708.071690] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4708.071692] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4708.071694] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4708.071697] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4708.071700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4708.071702] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4708.071709] Call Trace: [ 4708.071728] [ 4708.071734] ? show_regs.cold.9+0x22/0x2f [ 4708.071743] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4708.071749] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4708.071753] ? watchdog+0x40/0x40 [ 4708.071758] ? hrtimer_interrupt+0x128/0x2c0 [ 4708.071762] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4708.071771] ? apic_timer_interrupt+0xf/0x20 [ 4708.071775] [ 4708.071782] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4708.071787] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4708.071791] do_raw_spin_lock+0x177/0x190 [ 4708.071795] _raw_spin_lock+0x21/0x30 [ 4708.071801] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4708.080259] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4708.080481] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4708.080489] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4708.080491] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4708.080504] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4708.080508] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4708.080512] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4708.080515] RDX: 000000000000238f RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4708.080517] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4708.080519] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4708.080521] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4708.080524] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4708.080527] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4708.130000] ? do_raw_spin_unlock+0x75/0x190 [ 4708.130018] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4708.130714] ? do_wait_intr+0xf0/0xf0 [ 4708.130723] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4708.130932] mdc_reint+0x2d/0x110 [mdc] [ 4708.130971] mdc_unlink+0x658/0x870 [mdc] [ 4708.130999] lmv_unlink+0x34d/0xbd0 [lmv] [ 4708.131027] ll_unlink+0x24d/0x7f0 [lustre] [ 4708.138767] vfs_unlink+0x202/0x330 [ 4708.142631] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4708.142647] Call Trace: [ 4708.142655] [ 4708.142661] ? show_regs.cold.9+0x22/0x2f [ 4708.142675] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4708.142681] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4708.142685] ? watchdog+0x40/0x40 [ 4708.142689] ? hrtimer_interrupt+0x128/0x2c0 [ 4708.142693] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4708.142700] ? apic_timer_interrupt+0xf/0x20 [ 4708.142704] [ 4708.142717] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4708.142725] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4708.142729] do_raw_spin_lock+0x177/0x190 [ 4708.142734] _raw_spin_lock+0x21/0x30 [ 4708.142741] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4708.143265] obd_mod_free+0x46/0x140 [ptlrpc] [ 4708.151207] do_unlinkat+0x2fa/0x460 [ 4708.151224] __x64_sys_unlink+0x37/0x50 [ 4708.151229] do_syscall_64+0xc1/0x440 [ 4708.151236] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4708.151244] RIP: 0033:0x7fe02c11910b [ 4708.151252] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4708.151256] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4708.151262] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4708.151264] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4708.151265] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4708.151267] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4708.151269] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4708.704907] mdc_commit_open+0x91/0xb0 [mdc] [ 4708.706039] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4708.715104] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4708.720580] after_reply+0xe2c/0x1780 [ptlrpc] [ 4708.725236] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4708.739070] ? __next_timer_interrupt+0x160/0x160 [ 4708.740764] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4708.750395] ? do_wait_intr+0xf0/0xf0 [ 4708.755212] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4708.761384] kthread+0x1d1/0x200 [ 4708.771689] ? set_kthread_struct+0x70/0x70 [ 4708.777513] ret_from_fork+0x1f/0x30 [ 4716.069158] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4716.080965] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4716.142551] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4716.155166] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4716.161991] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4716.165257] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4716.175514] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4716.207149] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4716.217079] RAX: 0000000000007b30 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4716.220450] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4716.228704] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4716.231313] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4716.234999] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4716.240597] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4716.259651] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4716.273012] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4716.279202] Call Trace: [ 4716.281218] [ 4716.282265] ? show_regs.cold.9+0x22/0x2f [ 4716.286509] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4716.288982] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4716.291952] ? watchdog+0x40/0x40 [ 4716.293967] ? hrtimer_interrupt+0x128/0x2c0 [ 4716.297253] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4716.301804] ? apic_timer_interrupt+0xf/0x20 [ 4716.304289] [ 4716.305320] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4716.318823] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4716.325141] do_raw_spin_lock+0x177/0x190 [ 4716.326704] _raw_spin_lock+0x21/0x30 [ 4716.328728] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4716.337363] process_one_work+0x2c8/0x700 [ 4716.345649] worker_thread+0x296/0x6e0 [ 4716.350288] ? rescuer_thread+0x570/0x570 [ 4716.355789] kthread+0x1d1/0x200 [ 4716.366213] ? set_kthread_struct+0x70/0x70 [ 4716.370987] ret_from_fork+0x1f/0x30 [ 4722.149145] rcu: INFO: rcu_sched self-detected stall on CPU [ 4722.151254] rcu: 1-....: (1760439 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=501110 [ 4722.156052] (t=2580057 jiffies g=278549 q=13897) [ 4722.157815] Sending NMI from CPU 1 to CPUs 0: [ 4722.162837] NMI backtrace for cpu 0 [ 4722.162844] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4722.162846] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4722.162848] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4722.162849] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4722.162852] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 [ 4722.162855] RAX: 0000000000007ebe RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4722.162856] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4722.162857] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4722.162858] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4722.162860] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4722.162861] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4722.162862] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4722.162864] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4722.162865] Call Trace: [ 4722.162866] [ 4722.162866] ? show_regs.cold.9+0x22/0x2f [ 4722.162867] ? nmi_cpu_backtrace.cold.8+0x98/0xa2 [ 4722.162868] ? nmi_cpu_backtrace_handler+0x15/0x40 [ 4722.162869] ? nmi_handle+0xbd/0x210 [ 4722.162870] ? default_do_nmi+0x4b/0x1b0 [ 4722.162871] ? do_nmi+0x280/0x380 [ 4722.162872] ? end_repeat_nmi+0x16/0x1f [ 4722.162873] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4722.162874] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4722.162875] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4722.162876] [ 4722.162877] do_raw_spin_lock+0x177/0x190 [ 4722.162878] _raw_spin_lock+0x21/0x30 [ 4722.162878] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4722.162879] ? do_raw_spin_unlock+0x75/0x190 [ 4722.162880] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4722.162881] ? do_wait_intr+0xf0/0xf0 [ 4722.162882] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4722.162883] mdc_reint+0x2d/0x110 [mdc] [ 4722.162884] mdc_unlink+0x658/0x870 [mdc] [ 4722.162885] lmv_unlink+0x34d/0xbd0 [lmv] [ 4722.162886] ll_unlink+0x24d/0x7f0 [lustre] [ 4722.162887] vfs_unlink+0x202/0x330 [ 4722.162888] do_unlinkat+0x2fa/0x460 [ 4722.162889] __x64_sys_unlink+0x37/0x50 [ 4722.162890] do_syscall_64+0xc1/0x440 [ 4722.162891] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4722.162892] RIP: 0033:0x7fe02c11910b [ 4722.162893] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4722.162895] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4722.162897] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4722.162908] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4722.162909] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4722.162911] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4722.162912] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4722.167731] NMI backtrace for cpu 1 [ 4722.298793] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4722.303601] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4722.307452] Call Trace: [ 4722.308560] [ 4722.309516] ? dump_stack+0xbb/0x10e [ 4722.311196] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4722.313055] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4722.315079] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4722.317434] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4722.318873] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4722.320377] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4722.322208] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4722.324498] ? update_process_times+0x6d/0xc0 [ 4722.326271] ? tick_sched_handle.isra.18+0x27/0x90 [ 4722.328637] ? tick_sched_timer+0x53/0xb0 [ 4722.330428] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4722.332487] ? tick_sched_handle.isra.18+0x90/0x90 [ 4722.334550] ? hrtimer_interrupt+0x128/0x2c0 [ 4722.336410] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4722.338465] ? apic_timer_interrupt+0xf/0x20 [ 4722.340046] [ 4722.340833] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4722.343227] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4722.345517] ? do_raw_spin_lock+0x177/0x190 [ 4722.347280] ? _raw_spin_lock+0x21/0x30 [ 4722.349404] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4722.355828] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4722.358480] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4722.361116] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4722.363891] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4722.367279] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4722.370159] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4722.372728] ? __next_timer_interrupt+0x160/0x160 [ 4722.374510] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4722.376363] ? do_wait_intr+0xf0/0xf0 [ 4722.377383] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4722.380097] ? kthread+0x1d1/0x200 [ 4722.381145] ? set_kthread_struct+0x70/0x70 [ 4722.383149] ? ret_from_fork+0x1f/0x30 [ 4744.064663] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4744.068727] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4744.104718] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4744.111494] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4744.122815] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4744.127842] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4744.129672] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4744.135789] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4744.138232] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4744.140314] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4744.142034] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4744.143848] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4744.146613] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4744.149484] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4744.151850] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4744.153677] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4744.157096] Call Trace: [ 4744.157808] [ 4744.158440] ? show_regs.cold.9+0x22/0x2f [ 4744.159546] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4744.161408] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4744.162828] ? watchdog+0x40/0x40 [ 4744.164340] ? hrtimer_interrupt+0x128/0x2c0 [ 4744.166220] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4744.168256] ? apic_timer_interrupt+0xf/0x20 [ 4744.169879] [ 4744.170706] ? native_safe_halt+0x2e/0x50 [ 4744.172261] kvm_wait+0xcd/0xe0 [ 4744.173310] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4744.175119] do_raw_spin_lock+0x177/0x190 [ 4744.176725] _raw_spin_lock+0x21/0x30 [ 4744.178267] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4744.180970] process_one_work+0x2c8/0x700 [ 4744.182687] worker_thread+0x296/0x6e0 [ 4744.184439] ? rescuer_thread+0x570/0x570 [ 4744.186131] kthread+0x1d1/0x200 [ 4744.187450] ? set_kthread_struct+0x70/0x70 [ 4744.188963] ret_from_fork+0x1f/0x30 [ 4748.059846] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4748.061567] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4748.063643] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4748.063716] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4748.063723] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4748.063726] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4748.063736] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4748.063739] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4748.063743] RAX: 0000000000003b8d RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4748.063744] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4748.063746] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4748.063747] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4748.063749] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4748.063751] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4748.063753] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4748.063754] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4748.063761] Call Trace: [ 4748.063765] [ 4748.063769] ? show_regs.cold.9+0x22/0x2f [ 4748.063776] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4748.063780] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4748.063783] ? watchdog+0x40/0x40 [ 4748.063785] ? hrtimer_interrupt+0x128/0x2c0 [ 4748.063788] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4748.063794] ? apic_timer_interrupt+0xf/0x20 [ 4748.063797] [ 4748.063799] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4748.063802] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4748.063805] do_raw_spin_lock+0x177/0x190 [ 4748.063808] _raw_spin_lock+0x21/0x30 [ 4748.063811] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4748.064008] ? do_raw_spin_unlock+0x75/0x190 [ 4748.064011] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4748.064133] ? do_wait_intr+0xf0/0xf0 [ 4748.064139] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4748.064329] mdc_reint+0x2d/0x110 [mdc] [ 4748.064716] mdc_unlink+0x658/0x870 [mdc] [ 4748.064745] lmv_unlink+0x34d/0xbd0 [lmv] [ 4748.064775] ll_unlink+0x24d/0x7f0 [lustre] [ 4748.064933] vfs_unlink+0x202/0x330 [ 4748.064941] do_unlinkat+0x2fa/0x460 [ 4748.064946] __x64_sys_unlink+0x37/0x50 [ 4748.064949] do_syscall_64+0xc1/0x440 [ 4748.064955] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4748.064959] RIP: 0033:0x7fe02c11910b [ 4748.064965] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4748.064968] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4748.064972] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4748.064974] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4748.064976] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4748.064977] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4748.064979] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4748.278515] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4748.312804] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4748.317284] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4748.321595] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4748.324384] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4748.332411] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4748.336427] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4748.343568] RDX: 0000000000003fc0 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4748.348563] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4748.351379] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4748.360611] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4748.367591] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4748.376199] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4748.381210] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4748.387979] Call Trace: [ 4748.393189] [ 4748.394098] ? show_regs.cold.9+0x22/0x2f [ 4748.395642] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4748.400975] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4748.403457] ? watchdog+0x40/0x40 [ 4748.407227] ? hrtimer_interrupt+0x128/0x2c0 [ 4748.409582] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4748.414106] ? apic_timer_interrupt+0xf/0x20 [ 4748.418252] [ 4748.421404] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4748.427676] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4748.429864] do_raw_spin_lock+0x177/0x190 [ 4748.431601] _raw_spin_lock+0x21/0x30 [ 4748.433844] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4748.435451] obd_mod_free+0x46/0x140 [ptlrpc] [ 4748.440062] mdc_commit_open+0x91/0xb0 [mdc] [ 4748.444262] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4748.449287] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4748.454139] after_reply+0xe2c/0x1780 [ptlrpc] [ 4748.460228] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4748.463064] ? __next_timer_interrupt+0x160/0x160 [ 4748.466126] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4748.472809] ? do_wait_intr+0xf0/0xf0 [ 4748.477288] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4748.481034] kthread+0x1d1/0x200 [ 4748.482094] ? set_kthread_struct+0x70/0x70 [ 4748.491905] ret_from_fork+0x1f/0x30 [ 4772.068819] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 4772.076840] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4772.140249] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4772.154141] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4772.159302] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4772.166480] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4772.170549] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4772.182021] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4772.190816] RAX: 0000000000007e96 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4772.199909] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4772.202098] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4772.214522] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4772.221345] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4772.223355] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4772.233168] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4772.236503] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4772.254600] Call Trace: [ 4772.257383] [ 4772.258587] ? show_regs.cold.9+0x22/0x2f [ 4772.261503] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4772.268744] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4772.270011] ? watchdog+0x40/0x40 [ 4772.273002] ? hrtimer_interrupt+0x128/0x2c0 [ 4772.275209] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4772.278511] ? apic_timer_interrupt+0xf/0x20 [ 4772.282198] [ 4772.283251] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4772.285803] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4772.287375] do_raw_spin_lock+0x177/0x190 [ 4772.289793] _raw_spin_lock+0x21/0x30 [ 4772.291010] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4772.294949] process_one_work+0x2c8/0x700 [ 4772.300687] worker_thread+0x296/0x6e0 [ 4772.302889] ? rescuer_thread+0x570/0x570 [ 4772.306552] kthread+0x1d1/0x200 [ 4772.309007] ? set_kthread_struct+0x70/0x70 [ 4772.313371] ret_from_fork+0x1f/0x30 [ 4776.057027] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4776.061455] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4776.063396] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4776.105061] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4776.105080] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4776.105083] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4776.105150] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4776.105154] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4776.105161] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4776.105164] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4776.105165] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4776.105168] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4776.105170] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4776.105174] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4776.105177] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4776.105179] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4776.105187] Call Trace: [ 4776.105203] [ 4776.105211] ? show_regs.cold.9+0x22/0x2f [ 4776.105223] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4776.105230] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4776.105235] ? watchdog+0x40/0x40 [ 4776.105238] ? hrtimer_interrupt+0x128/0x2c0 [ 4776.105242] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4776.105249] ? apic_timer_interrupt+0xf/0x20 [ 4776.105253] [ 4776.105256] ? native_safe_halt+0x2e/0x50 [ 4776.105260] kvm_wait+0xcd/0xe0 [ 4776.105270] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4776.105279] do_raw_spin_lock+0x177/0x190 [ 4776.105283] _raw_spin_lock+0x21/0x30 [ 4776.105292] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4776.111300] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4776.111440] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4776.111448] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4776.111451] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4776.111466] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4776.111471] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4776.111475] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4776.111478] RDX: 0000000000007ebd RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4776.111480] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4776.111482] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4776.111484] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4776.111488] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4776.111491] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4776.111493] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4776.111500] Call Trace: [ 4776.111506] [ 4776.111512] ? show_regs.cold.9+0x22/0x2f [ 4776.111521] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4776.111526] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4776.111531] ? watchdog+0x40/0x40 [ 4776.111535] ? hrtimer_interrupt+0x128/0x2c0 [ 4776.111540] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4776.111547] ? apic_timer_interrupt+0xf/0x20 [ 4776.111552] [ 4776.111554] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4776.111559] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4776.111563] do_raw_spin_lock+0x177/0x190 [ 4776.111568] _raw_spin_lock+0x21/0x30 [ 4776.111574] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4776.111845] obd_mod_free+0x46/0x140 [ptlrpc] [ 4776.112112] mdc_commit_open+0x91/0xb0 [mdc] [ 4776.112150] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4776.112411] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4776.112599] after_reply+0xe2c/0x1780 [ptlrpc] [ 4776.112799] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4776.113066] ? __next_timer_interrupt+0x160/0x160 [ 4776.113076] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4776.113284] ? do_wait_intr+0xf0/0xf0 [ 4776.113293] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4776.113750] kthread+0x1d1/0x200 [ 4776.113756] ? set_kthread_struct+0x70/0x70 [ 4776.113760] ret_from_fork+0x1f/0x30 [ 4776.542800] ? do_raw_spin_unlock+0x75/0x190 [ 4776.549451] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4776.556287] ? do_wait_intr+0xf0/0xf0 [ 4776.561214] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4776.568464] mdc_reint+0x2d/0x110 [mdc] [ 4776.569652] mdc_unlink+0x658/0x870 [mdc] [ 4776.570672] lmv_unlink+0x34d/0xbd0 [lmv] [ 4776.578435] ll_unlink+0x24d/0x7f0 [lustre] [ 4776.582783] vfs_unlink+0x202/0x330 [ 4776.590932] do_unlinkat+0x2fa/0x460 [ 4776.592749] __x64_sys_unlink+0x37/0x50 [ 4776.594809] do_syscall_64+0xc1/0x440 [ 4776.596057] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4776.598779] RIP: 0033:0x7fe02c11910b [ 4776.600251] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4776.622613] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4776.627616] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4776.631272] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4776.644802] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4776.648973] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4776.654447] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4800.064046] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4800.068171] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4800.090594] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4800.095292] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4800.098685] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4800.101618] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4800.104365] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4800.112062] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4800.114297] RAX: 0000000000005a73 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4800.117590] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4800.121056] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4800.123708] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4800.126628] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4800.129857] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4800.133119] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4800.135347] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4800.138457] Call Trace: [ 4800.139409] [ 4800.140393] ? show_regs.cold.9+0x22/0x2f [ 4800.142144] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4800.144140] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4800.145980] ? watchdog+0x40/0x40 [ 4800.147235] ? hrtimer_interrupt+0x128/0x2c0 [ 4800.148916] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4800.151251] ? apic_timer_interrupt+0xf/0x20 [ 4800.153215] [ 4800.154078] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4800.156093] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4800.157980] do_raw_spin_lock+0x177/0x190 [ 4800.159402] _raw_spin_lock+0x21/0x30 [ 4800.160737] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4800.163053] process_one_work+0x2c8/0x700 [ 4800.164525] worker_thread+0x296/0x6e0 [ 4800.166179] ? rescuer_thread+0x570/0x570 [ 4800.168091] kthread+0x1d1/0x200 [ 4800.169634] ? set_kthread_struct+0x70/0x70 [ 4800.171577] ret_from_fork+0x1f/0x30 [ 4804.057980] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4804.060487] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4804.065665] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4804.065737] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4804.065745] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4804.065747] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4804.065761] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4804.065766] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4804.065770] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4804.065773] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4804.065775] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4804.065776] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4804.065793] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4804.065797] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4804.065800] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4804.065802] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4804.065809] Call Trace: [ 4804.065815] [ 4804.065820] ? show_regs.cold.9+0x22/0x2f [ 4804.065831] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4804.065838] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4804.065842] ? watchdog+0x40/0x40 [ 4804.065846] ? hrtimer_interrupt+0x128/0x2c0 [ 4804.065850] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4804.065858] ? apic_timer_interrupt+0xf/0x20 [ 4804.065863] [ 4804.066530] ? native_safe_halt+0x2e/0x50 [ 4804.066538] kvm_wait+0xcd/0xe0 [ 4804.066547] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4804.066554] do_raw_spin_lock+0x177/0x190 [ 4804.066559] _raw_spin_lock+0x21/0x30 [ 4804.066564] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4804.067189] ? do_raw_spin_unlock+0x75/0x190 [ 4804.067706] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4804.079446] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4804.082309] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4804.082317] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4804.082320] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4804.082334] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4804.082339] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4804.082343] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4804.082346] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4804.082348] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4804.082349] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4804.082351] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4804.082354] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4804.082357] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4804.082359] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4804.082367] Call Trace: [ 4804.082372] [ 4804.082377] ? show_regs.cold.9+0x22/0x2f [ 4804.082387] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4804.082394] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4804.082399] ? watchdog+0x40/0x40 [ 4804.082402] ? hrtimer_interrupt+0x128/0x2c0 [ 4804.082406] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4804.082412] ? apic_timer_interrupt+0xf/0x20 [ 4804.082416] [ 4804.082419] ? native_safe_halt+0x2e/0x50 [ 4804.082422] kvm_wait+0xcd/0xe0 [ 4804.082429] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4804.082435] do_raw_spin_lock+0x177/0x190 [ 4804.082440] _raw_spin_lock+0x21/0x30 [ 4804.082446] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4804.082717] obd_mod_free+0x46/0x140 [ptlrpc] [ 4804.084232] mdc_commit_open+0x91/0xb0 [mdc] [ 4804.084266] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4804.084477] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4804.084684] after_reply+0xe2c/0x1780 [ptlrpc] [ 4804.086043] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4804.087009] ? __next_timer_interrupt+0x160/0x160 [ 4804.087020] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4804.127503] ? do_wait_intr+0xf0/0xf0 [ 4804.127523] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4804.127788] mdc_reint+0x2d/0x110 [mdc] [ 4804.127827] mdc_unlink+0x658/0x870 [mdc] [ 4804.136371] ? do_wait_intr+0xf0/0xf0 [ 4804.136393] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4804.136680] kthread+0x1d1/0x200 [ 4804.136686] ? set_kthread_struct+0x70/0x70 [ 4804.136690] ret_from_fork+0x1f/0x30 [ 4804.379617] lmv_unlink+0x34d/0xbd0 [lmv] [ 4804.381426] ll_unlink+0x24d/0x7f0 [lustre] [ 4804.383702] vfs_unlink+0x202/0x330 [ 4804.385193] do_unlinkat+0x2fa/0x460 [ 4804.388802] __x64_sys_unlink+0x37/0x50 [ 4804.391263] do_syscall_64+0xc1/0x440 [ 4804.392589] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4804.394258] RIP: 0033:0x7fe02c11910b [ 4804.400002] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4804.413037] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4804.417261] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4804.427054] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4804.434050] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4804.437971] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4804.444131] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4828.070384] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4828.076385] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4828.119158] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4828.131979] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4828.142768] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4828.146626] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4828.151070] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4828.165602] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4828.171124] RAX: 0000000000007585 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4828.175426] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4828.180381] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4828.185009] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4828.196607] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4828.203866] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4828.211111] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4828.219628] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4828.228249] Call Trace: [ 4828.229155] [ 4828.232352] ? show_regs.cold.9+0x22/0x2f [ 4828.234174] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4828.241275] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4828.244949] ? watchdog+0x40/0x40 [ 4828.249108] ? hrtimer_interrupt+0x128/0x2c0 [ 4828.257181] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4828.260210] ? apic_timer_interrupt+0xf/0x20 [ 4828.262634] [ 4828.263330] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4828.268692] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4828.272553] do_raw_spin_lock+0x177/0x190 [ 4828.279722] _raw_spin_lock+0x21/0x30 [ 4828.281956] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4828.290813] process_one_work+0x2c8/0x700 [ 4828.296129] worker_thread+0x296/0x6e0 [ 4828.299474] ? rescuer_thread+0x570/0x570 [ 4828.300919] kthread+0x1d1/0x200 [ 4828.302692] ? set_kthread_struct+0x70/0x70 [ 4828.304027] ret_from_fork+0x1f/0x30 [ 4832.059433] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4832.063448] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4832.071190] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4832.071269] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4832.071278] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4832.071281] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4832.071296] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4832.071301] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4832.071305] RAX: 0000000000007114 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4832.071307] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4832.071309] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4832.071311] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4832.071313] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4832.071316] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4832.071319] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4832.071321] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4832.071330] Call Trace: [ 4832.071335] [ 4832.071341] ? show_regs.cold.9+0x22/0x2f [ 4832.071351] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4832.073307] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4832.073318] ? watchdog+0x40/0x40 [ 4832.073324] ? hrtimer_interrupt+0x128/0x2c0 [ 4832.073328] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4832.073337] ? apic_timer_interrupt+0xf/0x20 [ 4832.073341] [ 4832.073345] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4832.073351] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4832.073356] do_raw_spin_lock+0x177/0x190 [ 4832.074140] _raw_spin_lock+0x21/0x30 [ 4832.083123] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4832.083212] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4832.083219] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4832.083221] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4832.083236] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4832.083240] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4832.083244] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4832.083247] RDX: 000000000000466d RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4832.083249] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4832.083251] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4832.083254] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4832.083258] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4832.083261] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4832.083263] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4832.083272] Call Trace: [ 4832.083277] [ 4832.083283] ? show_regs.cold.9+0x22/0x2f [ 4832.083304] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4832.083311] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4832.083322] ? watchdog+0x40/0x40 [ 4832.083326] ? hrtimer_interrupt+0x128/0x2c0 [ 4832.083331] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4832.083339] ? apic_timer_interrupt+0xf/0x20 [ 4832.083344] [ 4832.083347] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4832.083352] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4832.083356] do_raw_spin_lock+0x177/0x190 [ 4832.083361] _raw_spin_lock+0x21/0x30 [ 4832.083366] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4832.083667] obd_mod_free+0x46/0x140 [ptlrpc] [ 4832.125617] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4832.126151] ? do_raw_spin_unlock+0x75/0x190 [ 4832.126165] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4832.129444] ? do_wait_intr+0xf0/0xf0 [ 4832.129459] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4832.155231] mdc_commit_open+0x91/0xb0 [mdc] [ 4832.166842] mdc_reint+0x2d/0x110 [mdc] [ 4832.170481] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4832.170829] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4832.186847] mdc_unlink+0x658/0x870 [mdc] [ 4832.190667] after_reply+0xe2c/0x1780 [ptlrpc] [ 4832.191044] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4832.191522] ? __next_timer_interrupt+0x160/0x160 [ 4832.191536] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4832.192125] ? do_wait_intr+0xf0/0xf0 [ 4832.192134] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4832.192536] kthread+0x1d1/0x200 [ 4832.192542] ? set_kthread_struct+0x70/0x70 [ 4832.192550] ret_from_fork+0x1f/0x30 [ 4832.524957] lmv_unlink+0x34d/0xbd0 [lmv] [ 4832.531474] ll_unlink+0x24d/0x7f0 [lustre] [ 4832.533500] vfs_unlink+0x202/0x330 [ 4832.535182] do_unlinkat+0x2fa/0x460 [ 4832.537698] __x64_sys_unlink+0x37/0x50 [ 4832.539044] do_syscall_64+0xc1/0x440 [ 4832.542146] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4832.544708] RIP: 0033:0x7fe02c11910b [ 4832.545938] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4832.555905] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4832.560090] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4832.564512] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4832.568193] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4832.570745] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4832.575633] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4856.064022] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4856.068059] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4856.087626] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4856.092435] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4856.096704] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4856.099749] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4856.102351] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4856.109452] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4856.112651] RAX: 000000000000590c RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4856.116896] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4856.121050] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4856.128872] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4856.134524] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4856.139022] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4856.144414] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4856.147667] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4856.149771] Call Trace: [ 4856.151111] [ 4856.151914] ? show_regs.cold.9+0x22/0x2f [ 4856.153495] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4856.155779] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4856.157657] ? watchdog+0x40/0x40 [ 4856.159010] ? hrtimer_interrupt+0x128/0x2c0 [ 4856.160986] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4856.162751] ? apic_timer_interrupt+0xf/0x20 [ 4856.164732] [ 4856.165588] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4856.167645] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4856.169921] do_raw_spin_lock+0x177/0x190 [ 4856.172088] _raw_spin_lock+0x21/0x30 [ 4856.174401] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4856.178081] process_one_work+0x2c8/0x700 [ 4856.181056] worker_thread+0x296/0x6e0 [ 4856.182620] ? rescuer_thread+0x570/0x570 [ 4856.186252] kthread+0x1d1/0x200 [ 4856.188748] ? set_kthread_struct+0x70/0x70 [ 4856.190540] ret_from_fork+0x1f/0x30 [ 4860.057109] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 4860.060543] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 4860.077157] Modules linked in: lustre(O) osp(O) [ 4860.082462] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4860.082557] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4860.082564] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4860.082567] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4860.082580] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4860.082585] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4860.082589] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4860.082592] RDX: 00000000000019b9 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4860.082594] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4860.082597] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4860.082599] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4860.082602] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4860.082605] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4860.082607] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4860.085153] Call Trace: [ 4860.089218] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4860.089293] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4860.089302] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4860.089304] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4860.089317] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4860.089321] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4860.089326] RAX: 000000000000450b RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4860.089328] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4860.089330] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4860.089333] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4860.089335] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4860.089338] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4860.089342] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4860.089344] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4860.089353] Call Trace: [ 4860.089358] [ 4860.089363] ? show_regs.cold.9+0x22/0x2f [ 4860.089372] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4860.089378] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4860.089384] ? watchdog+0x40/0x40 [ 4860.125845] [ 4860.125859] ? show_regs.cold.9+0x22/0x2f [ 4860.125885] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4860.125892] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4860.125896] ? watchdog+0x40/0x40 [ 4860.125900] ? hrtimer_interrupt+0x128/0x2c0 [ 4860.125903] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4860.125917] ? apic_timer_interrupt+0xf/0x20 [ 4860.125922] [ 4860.125925] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4860.125931] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4860.125936] do_raw_spin_lock+0x177/0x190 [ 4860.125948] _raw_spin_lock+0x21/0x30 [ 4860.125956] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4860.126438] obd_mod_free+0x46/0x140 [ptlrpc] [ 4860.126857] mdc_commit_open+0x91/0xb0 [mdc] [ 4860.126970] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4860.127553] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4860.127775] after_reply+0xe2c/0x1780 [ptlrpc] [ 4860.128130] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4860.128364] ? __next_timer_interrupt+0x160/0x160 [ 4860.128381] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4860.139026] ? hrtimer_interrupt+0x128/0x2c0 [ 4860.139047] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4860.139055] ? apic_timer_interrupt+0xf/0x20 [ 4860.139061] [ 4860.139067] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4860.139074] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4860.139078] do_raw_spin_lock+0x177/0x190 [ 4860.139085] _raw_spin_lock+0x21/0x30 [ 4860.139092] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4860.139364] ? do_raw_spin_unlock+0x75/0x190 [ 4860.139370] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4860.150349] ? do_wait_intr+0xf0/0xf0 [ 4860.150372] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4860.150672] kthread+0x1d1/0x200 [ 4860.150680] ? set_kthread_struct+0x70/0x70 [ 4860.150683] ret_from_fork+0x1f/0x30 [ 4860.380319] ? do_wait_intr+0xf0/0xf0 [ 4860.382679] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4860.385535] mdc_reint+0x2d/0x110 [mdc] [ 4860.387778] mdc_unlink+0x658/0x870 [mdc] [ 4860.390362] lmv_unlink+0x34d/0xbd0 [lmv] [ 4860.392412] ll_unlink+0x24d/0x7f0 [lustre] [ 4860.394704] vfs_unlink+0x202/0x330 [ 4860.396159] do_unlinkat+0x2fa/0x460 [ 4860.397528] __x64_sys_unlink+0x37/0x50 [ 4860.399737] do_syscall_64+0xc1/0x440 [ 4860.401474] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4860.404017] RIP: 0033:0x7fe02c11910b [ 4860.406785] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4860.415495] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4860.419234] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4860.422672] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4860.426859] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4860.431472] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4860.436980] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4884.064313] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4884.066971] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4884.103530] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4884.118629] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4884.125017] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4884.127998] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4884.129837] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4884.136445] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4884.138925] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4884.141061] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4884.143872] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4884.145881] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4884.147736] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4884.153029] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4884.157098] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4884.159861] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4884.162942] Call Trace: [ 4884.164119] [ 4884.165000] ? show_regs.cold.9+0x22/0x2f [ 4884.168421] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4884.171195] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4884.173551] ? watchdog+0x40/0x40 [ 4884.175851] ? hrtimer_interrupt+0x128/0x2c0 [ 4884.177882] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4884.180624] ? apic_timer_interrupt+0xf/0x20 [ 4884.184834] [ 4884.186435] ? native_safe_halt+0x2e/0x50 [ 4884.188913] kvm_wait+0xcd/0xe0 [ 4884.190083] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4884.192032] do_raw_spin_lock+0x177/0x190 [ 4884.194506] _raw_spin_lock+0x21/0x30 [ 4884.196180] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4884.198569] process_one_work+0x2c8/0x700 [ 4884.200213] worker_thread+0x296/0x6e0 [ 4884.202053] ? rescuer_thread+0x570/0x570 [ 4884.203856] kthread+0x1d1/0x200 [ 4884.205596] ? set_kthread_struct+0x70/0x70 [ 4884.207267] ret_from_fork+0x1f/0x30 [ 4888.057036] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4888.059786] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4888.060059] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4888.096129] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4888.100217] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4888.100476] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4888.100484] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4888.100486] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4888.100498] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4888.100502] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4888.100506] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4888.100509] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4888.100511] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4888.100513] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4888.100515] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4888.100518] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4888.100521] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4888.100523] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4888.100532] Call Trace: [ 4888.100537] [ 4888.100544] ? show_regs.cold.9+0x22/0x2f [ 4888.100622] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4888.100628] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4888.100632] ? watchdog+0x40/0x40 [ 4888.100636] ? hrtimer_interrupt+0x128/0x2c0 [ 4888.100639] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4888.100646] ? apic_timer_interrupt+0xf/0x20 [ 4888.100650] [ 4888.100653] ? native_safe_halt+0x2e/0x50 [ 4888.100656] kvm_wait+0xcd/0xe0 [ 4888.100664] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4888.100672] do_raw_spin_lock+0x177/0x190 [ 4888.100676] _raw_spin_lock+0x21/0x30 [ 4888.100680] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4888.101043] obd_mod_free+0x46/0x140 [ptlrpc] [ 4888.101238] mdc_commit_open+0x91/0xb0 [mdc] [ 4888.101275] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4888.119191] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4888.169846] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4888.170122] after_reply+0xe2c/0x1780 [ptlrpc] [ 4888.170248] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4888.170372] ? __next_timer_interrupt+0x160/0x160 [ 4888.170381] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4888.170513] ? do_wait_intr+0xf0/0xf0 [ 4888.170519] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4888.170723] kthread+0x1d1/0x200 [ 4888.170729] ? set_kthread_struct+0x70/0x70 [ 4888.170732] ret_from_fork+0x1f/0x30 [ 4888.381493] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4888.383114] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4888.395658] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 4888.403832] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4888.414213] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4888.419444] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4888.422629] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4888.426645] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4888.430059] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4888.434129] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4888.436907] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4888.439709] Call Trace: [ 4888.440932] [ 4888.441837] ? show_regs.cold.9+0x22/0x2f [ 4888.443836] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4888.445826] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4888.447671] ? watchdog+0x40/0x40 [ 4888.449089] ? hrtimer_interrupt+0x128/0x2c0 [ 4888.450871] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4888.452537] ? apic_timer_interrupt+0xf/0x20 [ 4888.453933] [ 4888.454651] ? native_safe_halt+0x2e/0x50 [ 4888.456130] kvm_wait+0xcd/0xe0 [ 4888.457531] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4888.459488] do_raw_spin_lock+0x177/0x190 [ 4888.460866] _raw_spin_lock+0x21/0x30 [ 4888.462038] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4888.464481] ? do_raw_spin_unlock+0x75/0x190 [ 4888.465894] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4888.475489] ? do_wait_intr+0xf0/0xf0 [ 4888.477697] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4888.481484] mdc_reint+0x2d/0x110 [mdc] [ 4888.484999] mdc_unlink+0x658/0x870 [mdc] [ 4888.487134] lmv_unlink+0x34d/0xbd0 [lmv] [ 4888.490537] ll_unlink+0x24d/0x7f0 [lustre] [ 4888.496469] vfs_unlink+0x202/0x330 [ 4888.497403] do_unlinkat+0x2fa/0x460 [ 4888.498400] __x64_sys_unlink+0x37/0x50 [ 4888.506958] do_syscall_64+0xc1/0x440 [ 4888.508015] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4888.511889] RIP: 0033:0x7fe02c11910b [ 4888.522901] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4888.549395] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4888.560503] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4888.570651] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4888.574137] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4888.590687] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4888.601093] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4902.152020] rcu: INFO: rcu_sched self-detected stall on CPU [ 4902.154395] rcu: 1-....: (1882726 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=536125 [ 4902.159390] (t=2760060 jiffies g=278549 q=14570) [ 4902.161587] Sending NMI from CPU 1 to CPUs 0: [ 4902.166166] NMI backtrace for cpu 0 [ 4902.166172] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4902.166174] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4902.166175] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4902.166177] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4902.166179] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 [ 4902.166181] RAX: 0000000000003c23 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4902.166183] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4902.166184] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4902.166185] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4902.166186] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4902.166187] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4902.166188] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4902.166189] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4902.166191] Call Trace: [ 4902.166191] [ 4902.166192] ? show_regs.cold.9+0x22/0x2f [ 4902.166193] ? nmi_cpu_backtrace.cold.8+0x98/0xa2 [ 4902.166194] ? nmi_cpu_backtrace_handler+0x15/0x40 [ 4902.166195] ? nmi_handle+0xbd/0x210 [ 4902.166196] ? default_do_nmi+0x4b/0x1b0 [ 4902.166197] ? do_nmi+0x280/0x380 [ 4902.166198] ? end_repeat_nmi+0x16/0x1f [ 4902.166199] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4902.166200] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4902.166202] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4902.166203] [ 4902.166204] do_raw_spin_lock+0x177/0x190 [ 4902.166205] _raw_spin_lock+0x21/0x30 [ 4902.166206] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4902.166207] ? do_raw_spin_unlock+0x75/0x190 [ 4902.166208] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4902.166209] ? do_wait_intr+0xf0/0xf0 [ 4902.166210] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4902.166211] mdc_reint+0x2d/0x110 [mdc] [ 4902.166212] mdc_unlink+0x658/0x870 [mdc] [ 4902.166213] lmv_unlink+0x34d/0xbd0 [lmv] [ 4902.166214] ll_unlink+0x24d/0x7f0 [lustre] [ 4902.166215] vfs_unlink+0x202/0x330 [ 4902.166216] do_unlinkat+0x2fa/0x460 [ 4902.166217] __x64_sys_unlink+0x37/0x50 [ 4902.166218] do_syscall_64+0xc1/0x440 [ 4902.166219] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4902.166220] RIP: 0033:0x7fe02c11910b [ 4902.166221] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4902.166223] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4902.166226] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4902.166227] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4902.166228] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4902.166230] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4902.166230] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4902.166985] NMI backtrace for cpu 1 [ 4902.311459] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4902.315074] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4902.318300] Call Trace: [ 4902.319202] [ 4902.319952] ? dump_stack+0xbb/0x10e [ 4902.321204] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4902.323471] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4902.325077] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4902.327312] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4902.329334] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4902.331271] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4902.333234] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4902.335319] ? update_process_times+0x6d/0xc0 [ 4902.336943] ? tick_sched_handle.isra.18+0x27/0x90 [ 4902.340743] ? tick_sched_timer+0x53/0xb0 [ 4902.342640] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4902.345045] ? tick_sched_handle.isra.18+0x90/0x90 [ 4902.349117] ? hrtimer_interrupt+0x128/0x2c0 [ 4902.351109] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4902.354534] ? apic_timer_interrupt+0xf/0x20 [ 4902.356168] [ 4902.357553] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4902.360719] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4902.363200] ? do_raw_spin_lock+0x177/0x190 [ 4902.365440] ? _raw_spin_lock+0x21/0x30 [ 4902.368332] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4902.372685] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4902.376571] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4902.379521] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4902.382664] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4902.385246] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4902.387871] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4902.391480] ? __next_timer_interrupt+0x160/0x160 [ 4902.393555] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4902.396128] ? do_wait_intr+0xf0/0xf0 [ 4902.398118] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4902.400975] ? kthread+0x1d1/0x200 [ 4902.403045] ? set_kthread_struct+0x70/0x70 [ 4902.405389] ? ret_from_fork+0x1f/0x30 [ 4912.064388] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4912.069261] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4912.118272] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4912.127215] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4912.146311] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4912.151603] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4912.159386] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4912.175188] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4912.182337] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4912.184019] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4912.192474] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4912.195779] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4912.201072] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4912.203491] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4912.212132] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4912.214119] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4912.225520] Call Trace: [ 4912.227592] [ 4912.230810] ? show_regs.cold.9+0x22/0x2f [ 4912.236948] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4912.246721] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4912.250824] ? watchdog+0x40/0x40 [ 4912.254836] ? hrtimer_interrupt+0x128/0x2c0 [ 4912.261858] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4912.267476] ? apic_timer_interrupt+0xf/0x20 [ 4912.269795] [ 4912.270875] ? native_safe_halt+0x2e/0x50 [ 4912.273394] kvm_wait+0xcd/0xe0 [ 4912.275026] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4912.278843] do_raw_spin_lock+0x177/0x190 [ 4912.281472] _raw_spin_lock+0x21/0x30 [ 4912.284052] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4912.289286] process_one_work+0x2c8/0x700 [ 4912.291289] worker_thread+0x296/0x6e0 [ 4912.293619] ? rescuer_thread+0x570/0x570 [ 4912.296777] kthread+0x1d1/0x200 [ 4912.300314] ? set_kthread_struct+0x70/0x70 [ 4912.305825] ret_from_fork+0x1f/0x30 [ 4928.057105] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4928.060049] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4928.061293] Modules linked in: lustre(O) [ 4928.064964] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4928.065174] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4928.065181] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4928.065186] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4928.065198] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4928.065202] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4928.065206] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4928.065209] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4928.065211] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4928.065213] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4928.065215] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4928.065218] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4928.065222] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4928.065224] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4928.065232] Call Trace: [ 4928.065238] [ 4928.065243] ? show_regs.cold.9+0x22/0x2f [ 4928.065253] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4928.065259] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4928.065264] ? watchdog+0x40/0x40 [ 4928.065268] ? hrtimer_interrupt+0x128/0x2c0 [ 4928.065274] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4928.065281] ? apic_timer_interrupt+0xf/0x20 [ 4928.065285] [ 4928.065288] ? native_safe_halt+0x2e/0x50 [ 4928.065291] kvm_wait+0xcd/0xe0 [ 4928.065311] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4928.065319] do_raw_spin_lock+0x177/0x190 [ 4928.065323] _raw_spin_lock+0x21/0x30 [ 4928.065328] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4928.065580] obd_mod_free+0x46/0x140 [ptlrpc] [ 4928.065834] mdc_commit_open+0x91/0xb0 [mdc] [ 4928.065876] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4928.068538] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4928.068606] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4928.068614] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4928.068617] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4928.068627] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4928.068631] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 4928.068636] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4928.068638] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4928.068640] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4928.068642] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4928.068644] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4928.068647] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4928.068650] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4928.068652] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4928.068660] Call Trace: [ 4928.068665] [ 4928.068670] ? show_regs.cold.9+0x22/0x2f [ 4928.068861] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4928.068870] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4928.068875] ? watchdog+0x40/0x40 [ 4928.068880] ? hrtimer_interrupt+0x128/0x2c0 [ 4928.068884] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4928.068891] ? apic_timer_interrupt+0xf/0x20 [ 4928.068895] [ 4928.068898] ? native_safe_halt+0x2e/0x50 [ 4928.068901] kvm_wait+0xcd/0xe0 [ 4928.068909] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4928.068915] do_raw_spin_lock+0x177/0x190 [ 4928.068920] _raw_spin_lock+0x21/0x30 [ 4928.068925] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4928.099305] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4928.107088] ? do_raw_spin_unlock+0x75/0x190 [ 4928.107176] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4928.107456] ? do_wait_intr+0xf0/0xf0 [ 4928.107464] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4928.107658] mdc_reint+0x2d/0x110 [mdc] [ 4928.115434] after_reply+0xe2c/0x1780 [ptlrpc] [ 4928.118531] mdc_unlink+0x658/0x870 [mdc] [ 4928.118579] lmv_unlink+0x34d/0xbd0 [lmv] [ 4928.118609] ll_unlink+0x24d/0x7f0 [lustre] [ 4928.133631] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4928.139350] vfs_unlink+0x202/0x330 [ 4928.139369] do_unlinkat+0x2fa/0x460 [ 4928.139374] __x64_sys_unlink+0x37/0x50 [ 4928.139378] do_syscall_64+0xc1/0x440 [ 4928.139386] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4928.139396] RIP: 0033:0x7fe02c11910b [ 4928.139405] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4928.139408] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4928.139414] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4928.139416] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4928.139418] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4928.139419] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4928.139421] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4928.405377] ? __next_timer_interrupt+0x160/0x160 [ 4928.407467] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4928.409541] ? do_wait_intr+0xf0/0xf0 [ 4928.411234] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4928.413609] kthread+0x1d1/0x200 [ 4928.415060] ? set_kthread_struct+0x70/0x70 [ 4928.416815] ret_from_fork+0x1f/0x30 [ 4940.066587] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4940.071287] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4940.098394] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4940.103127] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4940.107168] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4940.110435] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4940.114261] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4940.122126] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4940.124814] RAX: 000000000000420b RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4940.127600] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4940.130336] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4940.133170] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4940.136006] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4940.139173] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4940.142754] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4940.145065] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4940.148974] Call Trace: [ 4940.150256] [ 4940.151134] ? show_regs.cold.9+0x22/0x2f [ 4940.152954] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4940.155038] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4940.157027] ? watchdog+0x40/0x40 [ 4940.158658] ? hrtimer_interrupt+0x128/0x2c0 [ 4940.160260] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4940.162535] ? apic_timer_interrupt+0xf/0x20 [ 4940.164282] [ 4940.165313] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4940.167898] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4940.169609] do_raw_spin_lock+0x177/0x190 [ 4940.171418] _raw_spin_lock+0x21/0x30 [ 4940.172951] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4940.175637] process_one_work+0x2c8/0x700 [ 4940.177432] worker_thread+0x296/0x6e0 [ 4940.179009] ? rescuer_thread+0x570/0x570 [ 4940.180681] kthread+0x1d1/0x200 [ 4940.182156] ? set_kthread_struct+0x70/0x70 [ 4940.183906] ret_from_fork+0x1f/0x30 [ 4956.062740] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4956.064873] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4956.075919] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4956.076000] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4956.076007] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4956.076010] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4956.076022] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4956.076026] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4956.076030] RAX: 0000000000007f0a RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4956.076032] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 4956.076034] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4956.076036] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4956.076039] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4956.076042] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4956.076045] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4956.076047] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4956.076055] Call Trace: [ 4956.076060] [ 4956.076066] ? show_regs.cold.9+0x22/0x2f [ 4956.076076] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4956.076082] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4956.076086] ? watchdog+0x40/0x40 [ 4956.076090] ? hrtimer_interrupt+0x128/0x2c0 [ 4956.076093] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4956.076101] ? apic_timer_interrupt+0xf/0x20 [ 4956.076105] [ 4956.076108] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4956.076112] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4956.076117] do_raw_spin_lock+0x177/0x190 [ 4956.076122] _raw_spin_lock+0x21/0x30 [ 4956.076127] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4956.076978] ? do_raw_spin_unlock+0x75/0x190 [ 4956.076985] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4956.078272] ? do_wait_intr+0xf0/0xf0 [ 4956.078282] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4956.079120] mdc_reint+0x2d/0x110 [mdc] [ 4956.079159] mdc_unlink+0x658/0x870 [mdc] [ 4956.079889] lmv_unlink+0x34d/0xbd0 [lmv] [ 4956.079929] ll_unlink+0x24d/0x7f0 [lustre] [ 4956.080075] vfs_unlink+0x202/0x330 [ 4956.080085] do_unlinkat+0x2fa/0x460 [ 4956.080090] __x64_sys_unlink+0x37/0x50 [ 4956.080094] do_syscall_64+0xc1/0x440 [ 4956.080101] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4956.080111] RIP: 0033:0x7fe02c11910b [ 4956.080119] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4956.080123] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4956.080129] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4956.080131] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4956.080133] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4956.080136] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4956.080138] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4956.621834] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4956.681447] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4956.696174] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4956.712197] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4956.715822] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4956.727695] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4956.733109] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4956.737360] RDX: 000000000000601a RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4956.740782] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4956.747301] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4956.751349] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4956.756363] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4956.762390] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4956.766855] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4956.769731] Call Trace: [ 4956.772329] [ 4956.773778] ? show_regs.cold.9+0x22/0x2f [ 4956.776106] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4956.778679] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4956.781604] ? watchdog+0x40/0x40 [ 4956.783236] ? hrtimer_interrupt+0x128/0x2c0 [ 4956.784755] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4956.790532] ? apic_timer_interrupt+0xf/0x20 [ 4956.792386] [ 4956.793474] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4956.796082] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4956.802713] do_raw_spin_lock+0x177/0x190 [ 4956.804392] _raw_spin_lock+0x21/0x30 [ 4956.808951] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4956.813357] obd_mod_free+0x46/0x140 [ptlrpc] [ 4956.816828] mdc_commit_open+0x91/0xb0 [mdc] [ 4956.822358] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4956.826643] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4956.832087] after_reply+0xe2c/0x1780 [ptlrpc] [ 4956.837909] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4956.842065] ? __next_timer_interrupt+0x160/0x160 [ 4956.845135] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4956.847688] ? do_wait_intr+0xf0/0xf0 [ 4956.852071] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4956.855385] kthread+0x1d1/0x200 [ 4956.857535] ? set_kthread_struct+0x70/0x70 [ 4956.859684] ret_from_fork+0x1f/0x30 [ 4968.064174] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4968.071421] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4968.137851] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4968.143774] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4968.153707] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4968.156652] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4968.164780] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4968.184263] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4968.191771] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4968.194466] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4968.198374] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4968.202965] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4968.207703] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4968.213188] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4968.217742] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4968.221978] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4968.228422] Call Trace: [ 4968.230318] [ 4968.233814] ? show_regs.cold.9+0x22/0x2f [ 4968.240968] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4968.248216] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4968.250958] ? watchdog+0x40/0x40 [ 4968.260353] ? hrtimer_interrupt+0x128/0x2c0 [ 4968.262758] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4968.272798] ? apic_timer_interrupt+0xf/0x20 [ 4968.281718] [ 4968.283949] ? native_safe_halt+0x2e/0x50 [ 4968.289112] kvm_wait+0xcd/0xe0 [ 4968.293944] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4968.301984] do_raw_spin_lock+0x177/0x190 [ 4968.304163] _raw_spin_lock+0x21/0x30 [ 4968.305297] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4968.311779] process_one_work+0x2c8/0x700 [ 4968.317461] worker_thread+0x296/0x6e0 [ 4968.321154] ? rescuer_thread+0x570/0x570 [ 4968.323727] kthread+0x1d1/0x200 [ 4968.328496] ? set_kthread_struct+0x70/0x70 [ 4968.335104] ret_from_fork+0x1f/0x30 [ 4984.057019] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 4984.060491] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 4984.064560] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4984.064646] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4984.064653] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4984.064657] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4984.064669] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4984.064674] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4984.064679] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4984.064681] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 4984.064683] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 4984.064686] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 4984.064688] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 4984.064692] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 4984.064695] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4984.064698] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 4984.064707] Call Trace: [ 4984.064712] [ 4984.064718] ? show_regs.cold.9+0x22/0x2f [ 4984.064728] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4984.064735] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4984.064740] ? watchdog+0x40/0x40 [ 4984.064744] ? hrtimer_interrupt+0x128/0x2c0 [ 4984.064748] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4984.064755] ? apic_timer_interrupt+0xf/0x20 [ 4984.064760] [ 4984.064763] ? native_safe_halt+0x2e/0x50 [ 4984.064768] kvm_wait+0xcd/0xe0 [ 4984.064776] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4984.064784] do_raw_spin_lock+0x177/0x190 [ 4984.064788] _raw_spin_lock+0x21/0x30 [ 4984.064794] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4984.065150] ? do_raw_spin_unlock+0x75/0x190 [ 4984.065157] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4984.065497] ? do_wait_intr+0xf0/0xf0 [ 4984.065506] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4984.065721] mdc_reint+0x2d/0x110 [mdc] [ 4984.065760] mdc_unlink+0x658/0x870 [mdc] [ 4984.065786] lmv_unlink+0x34d/0xbd0 [lmv] [ 4984.065819] ll_unlink+0x24d/0x7f0 [lustre] [ 4984.065936] vfs_unlink+0x202/0x330 [ 4984.065944] do_unlinkat+0x2fa/0x460 [ 4984.065949] __x64_sys_unlink+0x37/0x50 [ 4984.065953] do_syscall_64+0xc1/0x440 [ 4984.065960] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4984.065967] RIP: 0033:0x7fe02c11910b [ 4984.065973] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4984.065977] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4984.065983] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 4984.065985] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 4984.065988] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 4984.065990] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 4984.065992] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4984.350189] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4984.416489] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4984.429848] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4984.443456] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4984.449331] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4984.488422] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4984.495592] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 4984.501058] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 4984.512555] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 4984.521820] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 4984.529334] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4984.533857] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 4984.537900] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4984.545980] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 4984.550453] Call Trace: [ 4984.551642] [ 4984.552983] ? show_regs.cold.9+0x22/0x2f [ 4984.554571] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4984.557881] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4984.567519] ? watchdog+0x40/0x40 [ 4984.573341] ? hrtimer_interrupt+0x128/0x2c0 [ 4984.581480] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4984.587641] ? apic_timer_interrupt+0xf/0x20 [ 4984.589060] [ 4984.589786] ? native_safe_halt+0x2e/0x50 [ 4984.595515] kvm_wait+0xcd/0xe0 [ 4984.596583] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4984.600809] do_raw_spin_lock+0x177/0x190 [ 4984.602920] _raw_spin_lock+0x21/0x30 [ 4984.610248] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4984.613569] obd_mod_free+0x46/0x140 [ptlrpc] [ 4984.623541] mdc_commit_open+0x91/0xb0 [mdc] [ 4984.627826] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4984.634276] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4984.640627] after_reply+0xe2c/0x1780 [ptlrpc] [ 4984.644230] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4984.651973] ? __next_timer_interrupt+0x160/0x160 [ 4984.654778] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4984.657124] ? do_wait_intr+0xf0/0xf0 [ 4984.658359] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4984.661469] kthread+0x1d1/0x200 [ 4984.665670] ? set_kthread_struct+0x70/0x70 [ 4984.669785] ret_from_fork+0x1f/0x30 [ 4996.064030] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 4996.068465] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4996.106191] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4996.113332] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4996.118490] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4996.121877] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4996.123428] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4996.131210] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 4996.136610] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 4996.139878] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 4996.147402] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4996.155114] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 4996.158453] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 4996.161000] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 4996.166193] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4996.168306] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 4996.172679] Call Trace: [ 4996.173822] [ 4996.174913] ? show_regs.cold.9+0x22/0x2f [ 4996.176778] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4996.178796] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4996.180865] ? watchdog+0x40/0x40 [ 4996.181996] ? hrtimer_interrupt+0x128/0x2c0 [ 4996.183661] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4996.185836] ? apic_timer_interrupt+0xf/0x20 [ 4996.187845] [ 4996.189023] ? native_safe_halt+0x2e/0x50 [ 4996.190676] kvm_wait+0xcd/0xe0 [ 4996.191802] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4996.193841] do_raw_spin_lock+0x177/0x190 [ 4996.195394] _raw_spin_lock+0x21/0x30 [ 4996.200637] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4996.204224] process_one_work+0x2c8/0x700 [ 4996.206211] worker_thread+0x296/0x6e0 [ 4996.207235] ? rescuer_thread+0x570/0x570 [ 4996.218218] kthread+0x1d1/0x200 [ 4996.220195] ? set_kthread_struct+0x70/0x70 [ 4996.223454] ret_from_fork+0x1f/0x30 [ 5012.057069] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5012.060021] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5012.062256] Modules linked in: lustre(O) [ 5012.065070] Modules linked in: [ 5012.065071] osp(O) ofd(O) [ 5012.066475] lustre(O) [ 5012.067914] lod(O) [ 5012.068961] osp(O) [ 5012.070581] mdt(O) [ 5012.071600] ofd(O) [ 5012.072443] mdd(O) [ 5012.073321] lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5012.073389] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5012.073395] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5012.073398] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5012.073412] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5012.073416] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5012.073421] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5012.073424] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5012.073426] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5012.073434] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5012.073436] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5012.073439] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5012.073442] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5012.073444] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5012.073452] Call Trace: [ 5012.073463] [ 5012.073468] ? show_regs.cold.9+0x22/0x2f [ 5012.074516] mgs(O) [ 5012.075519] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5012.105182] lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5012.105322] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5012.105329] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5012.105332] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5012.105345] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5012.105349] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5012.105354] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5012.105356] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 5012.105358] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5012.105361] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5012.105364] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5012.105367] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5012.105370] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5012.105373] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5012.105381] Call Trace: [ 5012.105387] [ 5012.105393] ? show_regs.cold.9+0x22/0x2f [ 5012.105402] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5012.105408] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5012.105413] ? watchdog+0x40/0x40 [ 5012.105418] ? hrtimer_interrupt+0x128/0x2c0 [ 5012.105422] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5012.105430] ? apic_timer_interrupt+0xf/0x20 [ 5012.105435] [ 5012.105438] ? native_safe_halt+0x2e/0x50 [ 5012.105442] kvm_wait+0xcd/0xe0 [ 5012.105449] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5012.105457] do_raw_spin_lock+0x177/0x190 [ 5012.105461] _raw_spin_lock+0x21/0x30 [ 5012.105466] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5012.105738] ? do_raw_spin_unlock+0x75/0x190 [ 5012.105745] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5012.105946] ? do_wait_intr+0xf0/0xf0 [ 5012.105955] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5012.106150] mdc_reint+0x2d/0x110 [mdc] [ 5012.106193] mdc_unlink+0x658/0x870 [mdc] [ 5012.106294] lmv_unlink+0x34d/0xbd0 [lmv] [ 5012.106327] ll_unlink+0x24d/0x7f0 [lustre] [ 5012.106450] vfs_unlink+0x202/0x330 [ 5012.106458] do_unlinkat+0x2fa/0x460 [ 5012.106463] __x64_sys_unlink+0x37/0x50 [ 5012.106467] do_syscall_64+0xc1/0x440 [ 5012.106473] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5012.106479] RIP: 0033:0x7fe02c11910b [ 5012.106486] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5012.106489] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5012.106495] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5012.106497] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5012.106499] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5012.106502] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5012.106504] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5012.345353] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5012.347387] ? watchdog+0x40/0x40 [ 5012.349840] ? hrtimer_interrupt+0x128/0x2c0 [ 5012.352049] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5012.355102] ? apic_timer_interrupt+0xf/0x20 [ 5012.357942] [ 5012.360347] ? native_safe_halt+0x2e/0x50 [ 5012.364516] kvm_wait+0xcd/0xe0 [ 5012.366950] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5012.374171] do_raw_spin_lock+0x177/0x190 [ 5012.378752] _raw_spin_lock+0x21/0x30 [ 5012.379986] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5012.384419] obd_mod_free+0x46/0x140 [ptlrpc] [ 5012.388096] mdc_commit_open+0x91/0xb0 [mdc] [ 5012.389676] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5012.392975] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5012.400162] after_reply+0xe2c/0x1780 [ptlrpc] [ 5012.403634] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5012.409451] ? __next_timer_interrupt+0x160/0x160 [ 5012.411607] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5012.413980] ? do_wait_intr+0xf0/0xf0 [ 5012.415574] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5012.417918] kthread+0x1d1/0x200 [ 5012.418941] ? set_kthread_struct+0x70/0x70 [ 5012.420771] ret_from_fork+0x1f/0x30 [ 5024.064030] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 5024.071266] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5024.089680] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5024.093763] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5024.097282] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5024.099729] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5024.101499] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5024.108740] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5024.111490] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5024.113506] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5024.116680] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5024.119503] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5024.122686] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5024.125650] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5024.128726] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5024.131152] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5024.133362] Call Trace: [ 5024.134231] [ 5024.135576] ? show_regs.cold.9+0x22/0x2f [ 5024.137253] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5024.139399] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5024.141252] ? watchdog+0x40/0x40 [ 5024.142655] ? hrtimer_interrupt+0x128/0x2c0 [ 5024.144138] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5024.145603] ? apic_timer_interrupt+0xf/0x20 [ 5024.147282] [ 5024.148183] ? native_safe_halt+0x2e/0x50 [ 5024.149747] kvm_wait+0xcd/0xe0 [ 5024.151194] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5024.153647] do_raw_spin_lock+0x177/0x190 [ 5024.155240] _raw_spin_lock+0x21/0x30 [ 5024.157061] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5024.159718] process_one_work+0x2c8/0x700 [ 5024.161498] worker_thread+0x296/0x6e0 [ 5024.162971] ? rescuer_thread+0x570/0x570 [ 5024.164913] kthread+0x1d1/0x200 [ 5024.166458] ? set_kthread_struct+0x70/0x70 [ 5024.168604] ret_from_fork+0x1f/0x30 [ 5040.057023] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5040.060035] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5040.060126] Modules linked in: [ 5040.062429] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5040.062505] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5040.062512] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5040.062515] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5040.063918] lustre(O) [ 5040.089891] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5040.089905] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5040.089912] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5040.089915] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5040.089917] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5040.089919] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5040.089922] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5040.089925] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5040.089928] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5040.089930] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5040.089939] Call Trace: [ 5040.089943] [ 5040.089949] ? show_regs.cold.9+0x22/0x2f [ 5040.094780] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5040.094859] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5040.094868] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5040.094870] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5040.094885] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5040.094890] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5040.094894] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5040.094897] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 5040.094899] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5040.094901] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5040.094904] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5040.094908] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5040.094911] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5040.094914] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5040.094922] Call Trace: [ 5040.094928] [ 5040.094934] ? show_regs.cold.9+0x22/0x2f [ 5040.094945] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5040.094952] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5040.094956] ? watchdog+0x40/0x40 [ 5040.094961] ? hrtimer_interrupt+0x128/0x2c0 [ 5040.094965] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5040.094971] ? apic_timer_interrupt+0xf/0x20 [ 5040.094975] [ 5040.094978] ? native_safe_halt+0x2e/0x50 [ 5040.094982] kvm_wait+0xcd/0xe0 [ 5040.094990] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5040.094997] do_raw_spin_lock+0x177/0x190 [ 5040.095002] _raw_spin_lock+0x21/0x30 [ 5040.095008] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5040.095359] ? do_raw_spin_unlock+0x75/0x190 [ 5040.095367] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5040.095641] ? do_wait_intr+0xf0/0xf0 [ 5040.095648] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5040.095908] mdc_reint+0x2d/0x110 [mdc] [ 5040.095949] mdc_unlink+0x658/0x870 [mdc] [ 5040.095978] lmv_unlink+0x34d/0xbd0 [lmv] [ 5040.096010] ll_unlink+0x24d/0x7f0 [lustre] [ 5040.103977] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5040.103997] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5040.104002] ? watchdog+0x40/0x40 [ 5040.104007] ? hrtimer_interrupt+0x128/0x2c0 [ 5040.104011] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5040.104019] ? apic_timer_interrupt+0xf/0x20 [ 5040.104024] [ 5040.104030] ? native_safe_halt+0x2e/0x50 [ 5040.104035] kvm_wait+0xcd/0xe0 [ 5040.104044] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5040.104049] do_raw_spin_lock+0x177/0x190 [ 5040.104052] _raw_spin_lock+0x21/0x30 [ 5040.104058] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5040.104309] obd_mod_free+0x46/0x140 [ptlrpc] [ 5040.104515] mdc_commit_open+0x91/0xb0 [mdc] [ 5040.104553] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5040.104745] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5040.104939] after_reply+0xe2c/0x1780 [ptlrpc] [ 5040.105129] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5040.105319] ? __next_timer_interrupt+0x160/0x160 [ 5040.105327] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5040.105556] ? do_wait_intr+0xf0/0xf0 [ 5040.105563] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5040.105758] kthread+0x1d1/0x200 [ 5040.105763] ? set_kthread_struct+0x70/0x70 [ 5040.105767] ret_from_fork+0x1f/0x30 [ 5040.287232] vfs_unlink+0x202/0x330 [ 5040.288504] do_unlinkat+0x2fa/0x460 [ 5040.289791] __x64_sys_unlink+0x37/0x50 [ 5040.291390] do_syscall_64+0xc1/0x440 [ 5040.292832] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5040.294636] RIP: 0033:0x7fe02c11910b [ 5040.295988] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5040.302433] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5040.304875] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5040.307422] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5040.309844] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5040.312345] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5040.314955] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5052.064116] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 5052.075450] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5052.123791] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5052.137643] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5052.149119] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5052.155995] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5052.161831] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5052.174539] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5052.180463] RAX: 0000000000007714 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5052.190859] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5052.201506] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5052.211107] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5052.219818] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5052.230071] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5052.233425] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5052.236968] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5052.240368] Call Trace: [ 5052.243267] [ 5052.243922] ? show_regs.cold.9+0x22/0x2f [ 5052.245005] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5052.255685] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5052.257332] ? watchdog+0x40/0x40 [ 5052.262032] ? hrtimer_interrupt+0x128/0x2c0 [ 5052.267712] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5052.270998] ? apic_timer_interrupt+0xf/0x20 [ 5052.276452] [ 5052.278351] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5052.283095] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5052.285455] do_raw_spin_lock+0x177/0x190 [ 5052.288349] _raw_spin_lock+0x21/0x30 [ 5052.291735] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5052.295488] process_one_work+0x2c8/0x700 [ 5052.299831] worker_thread+0x296/0x6e0 [ 5052.302779] ? rescuer_thread+0x570/0x570 [ 5052.305959] kthread+0x1d1/0x200 [ 5052.308188] ? set_kthread_struct+0x70/0x70 [ 5052.311825] ret_from_fork+0x1f/0x30 [ 5068.062686] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 5068.064575] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 5068.071932] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5068.072163] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5068.072173] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5068.072176] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5068.072197] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5068.072202] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5068.072322] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5068.072325] RDX: 0000000000006b4d RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5068.072327] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5068.072330] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5068.072332] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5068.072335] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5068.072338] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5068.072341] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5068.072350] Call Trace: [ 5068.072359] [ 5068.072570] ? show_regs.cold.9+0x22/0x2f [ 5068.072584] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5068.072592] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5068.072596] ? watchdog+0x40/0x40 [ 5068.072601] ? hrtimer_interrupt+0x128/0x2c0 [ 5068.072604] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5068.072614] ? apic_timer_interrupt+0xf/0x20 [ 5068.072619] [ 5068.072622] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5068.072627] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5068.072631] do_raw_spin_lock+0x177/0x190 [ 5068.072637] _raw_spin_lock+0x21/0x30 [ 5068.072644] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5068.073083] obd_mod_free+0x46/0x140 [ptlrpc] [ 5068.080616] Modules linked in: lustre(O) [ 5068.141167] mdc_commit_open+0x91/0xb0 [mdc] [ 5068.141220] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5068.141553] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5068.152608] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5068.152682] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5068.152689] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5068.152692] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5068.152707] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5068.152712] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5068.152716] RAX: 000000000000560d RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5068.152719] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5068.152721] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5068.152723] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5068.152726] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5068.152729] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5068.152732] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5068.152734] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5068.152744] Call Trace: [ 5068.152749] [ 5068.152754] ? show_regs.cold.9+0x22/0x2f [ 5068.152763] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5068.152770] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5068.152775] ? watchdog+0x40/0x40 [ 5068.152780] ? hrtimer_interrupt+0x128/0x2c0 [ 5068.152784] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5068.152792] ? apic_timer_interrupt+0xf/0x20 [ 5068.152797] [ 5068.152800] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5068.152805] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5068.152809] do_raw_spin_lock+0x177/0x190 [ 5068.152814] _raw_spin_lock+0x21/0x30 [ 5068.152820] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5068.153456] ? do_raw_spin_unlock+0x75/0x190 [ 5068.153463] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5068.153791] ? do_wait_intr+0xf0/0xf0 [ 5068.153799] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5068.154472] mdc_reint+0x2d/0x110 [mdc] [ 5068.154509] mdc_unlink+0x658/0x870 [mdc] [ 5068.154536] lmv_unlink+0x34d/0xbd0 [lmv] [ 5068.154626] ll_unlink+0x24d/0x7f0 [lustre] [ 5068.154759] vfs_unlink+0x202/0x330 [ 5068.154767] do_unlinkat+0x2fa/0x460 [ 5068.154771] __x64_sys_unlink+0x37/0x50 [ 5068.154774] do_syscall_64+0xc1/0x440 [ 5068.154781] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5068.154788] RIP: 0033:0x7fe02c11910b [ 5068.154795] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5068.154799] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5068.154804] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5068.154806] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5068.154808] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5068.154811] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5068.154813] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5068.651166] after_reply+0xe2c/0x1780 [ptlrpc] [ 5068.653915] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5068.657745] ? __next_timer_interrupt+0x160/0x160 [ 5068.660239] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5068.662782] ? do_wait_intr+0xf0/0xf0 [ 5068.664658] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5068.667347] kthread+0x1d1/0x200 [ 5068.668820] ? set_kthread_struct+0x70/0x70 [ 5068.670638] ret_from_fork+0x1f/0x30 [ 5080.066056] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 5080.089955] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5080.140953] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5080.149319] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5080.159003] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5080.165113] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5080.173244] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5080.190990] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5080.196399] RAX: 000000000000705b RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5080.208781] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5080.215794] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5080.224947] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5080.229051] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5080.239107] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5080.245788] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5080.251859] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5080.260211] Call Trace: [ 5080.263765] [ 5080.264504] ? show_regs.cold.9+0x22/0x2f [ 5080.272809] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5080.275853] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5080.279369] ? watchdog+0x40/0x40 [ 5080.283013] ? hrtimer_interrupt+0x128/0x2c0 [ 5080.287867] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5080.290771] ? apic_timer_interrupt+0xf/0x20 [ 5080.296586] [ 5080.299441] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5080.306593] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5080.308449] do_raw_spin_lock+0x177/0x190 [ 5080.311388] _raw_spin_lock+0x21/0x30 [ 5080.313705] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5080.318401] process_one_work+0x2c8/0x700 [ 5080.320717] worker_thread+0x296/0x6e0 [ 5080.324260] ? rescuer_thread+0x570/0x570 [ 5080.325578] kthread+0x1d1/0x200 [ 5080.331407] ? set_kthread_struct+0x70/0x70 [ 5080.335381] ret_from_fork+0x1f/0x30 [ 5082.162818] rcu: INFO: rcu_sched detected stalls on CPUs/tasks: [ 5082.175451] rcu: 0-....: (1970205 ticks this GP) idle=c12/1/0x4000000000000000 softirq=132993/132993 fqs=569910 [ 5082.188413] rcu: 1-....: (1995369 ticks this GP) idle=fe2/1/0x4000000000000000 softirq=133713/133713 fqs=569911 [ 5082.198905] (detected by 3, t=2940105 jiffies, g=278549, q=15389) [ 5082.203817] Sending NMI from CPU 3 to CPUs 0: [ 5082.206440] NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2e/0x50 [ 5082.208927] Sending NMI from CPU 3 to CPUs 1: [ 5082.227762] NMI backtrace for cpu 1 [ 5082.227769] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5082.227771] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5082.227773] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5082.227775] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5082.227778] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 [ 5082.227780] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5082.227782] RDX: 0000000000007dfc RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5082.227783] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5082.227784] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5082.227785] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5082.227786] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5082.227787] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5082.227789] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5082.227790] Call Trace: [ 5082.227790] [ 5082.227791] ? show_regs.cold.9+0x22/0x2f [ 5082.227792] ? nmi_cpu_backtrace.cold.8+0x98/0xa2 [ 5082.227793] ? nmi_cpu_backtrace_handler+0x15/0x40 [ 5082.227794] ? nmi_handle+0xbd/0x210 [ 5082.227795] ? default_do_nmi+0x4b/0x1b0 [ 5082.227796] ? do_nmi+0x280/0x380 [ 5082.227797] ? end_repeat_nmi+0x16/0x1f [ 5082.227797] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5082.227798] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5082.227799] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5082.227800] [ 5082.227801] do_raw_spin_lock+0x177/0x190 [ 5082.227802] _raw_spin_lock+0x21/0x30 [ 5082.227803] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5082.227803] obd_mod_free+0x46/0x140 [ptlrpc] [ 5082.227804] mdc_commit_open+0x91/0xb0 [mdc] [ 5082.227805] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5082.227806] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5082.227807] after_reply+0xe2c/0x1780 [ptlrpc] [ 5082.227808] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5082.227809] ? __next_timer_interrupt+0x160/0x160 [ 5082.227810] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5082.227810] ? do_wait_intr+0xf0/0xf0 [ 5082.228019] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5082.228021] kthread+0x1d1/0x200 [ 5082.228021] ? set_kthread_struct+0x70/0x70 [ 5082.228022] ret_from_fork+0x1f/0x30 [ 5108.057030] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 5108.060023] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 5108.064028] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 5108.064041] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5108.064113] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5108.064119] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5108.064126] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5108.064448] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5108.064466] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5108.064469] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5108.064474] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5108.064476] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5108.064478] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5108.064480] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5108.064482] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5108.064486] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5108.064489] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5108.064491] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5108.064499] Call Trace: [ 5108.064508] [ 5108.064513] ? show_regs.cold.9+0x22/0x2f [ 5108.064522] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5108.064529] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5108.064533] ? watchdog+0x40/0x40 [ 5108.064538] ? hrtimer_interrupt+0x128/0x2c0 [ 5108.064542] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5108.064548] ? apic_timer_interrupt+0xf/0x20 [ 5108.064552] [ 5108.064555] ? native_safe_halt+0x2e/0x50 [ 5108.064559] kvm_wait+0xcd/0xe0 [ 5108.064565] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5108.064572] do_raw_spin_lock+0x177/0x190 [ 5108.064576] _raw_spin_lock+0x21/0x30 [ 5108.064580] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5108.064803] process_one_work+0x2c8/0x700 [ 5108.064810] worker_thread+0x296/0x6e0 [ 5108.064815] ? rescuer_thread+0x570/0x570 [ 5108.064818] kthread+0x1d1/0x200 [ 5108.064823] ? set_kthread_struct+0x70/0x70 [ 5108.064826] ret_from_fork+0x1f/0x30 [ 5108.065790] Modules linked in: [ 5108.067756] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5108.067844] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5108.067852] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5108.067855] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5108.067870] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5108.067874] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5108.067879] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5108.067881] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5108.067883] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5108.067886] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5108.067889] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5108.067892] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5108.067896] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5108.067898] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5108.067906] Call Trace: [ 5108.067912] [ 5108.067918] ? show_regs.cold.9+0x22/0x2f [ 5108.067930] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5108.067937] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5108.067942] ? watchdog+0x40/0x40 [ 5108.067947] ? hrtimer_interrupt+0x128/0x2c0 [ 5108.067951] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5108.067959] ? apic_timer_interrupt+0xf/0x20 [ 5108.067963] [ 5108.067966] ? native_safe_halt+0x2e/0x50 [ 5108.067971] kvm_wait+0xcd/0xe0 [ 5108.067979] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5108.067988] do_raw_spin_lock+0x177/0x190 [ 5108.067993] _raw_spin_lock+0x21/0x30 [ 5108.067998] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5108.068338] obd_mod_free+0x46/0x140 [ptlrpc] [ 5108.068568] mdc_commit_open+0x91/0xb0 [mdc] [ 5108.068609] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5108.068835] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5108.069065] after_reply+0xe2c/0x1780 [ptlrpc] [ 5108.072053] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5108.072250] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5108.072259] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5108.072262] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5108.072278] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5108.072282] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5108.072287] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5108.072290] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 5108.072292] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5108.072295] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5108.072298] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5108.072302] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5108.072305] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5108.072307] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5108.072315] Call Trace: [ 5108.072321] [ 5108.072328] ? show_regs.cold.9+0x22/0x2f [ 5108.072339] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5108.072346] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5108.072350] ? watchdog+0x40/0x40 [ 5108.072355] ? hrtimer_interrupt+0x128/0x2c0 [ 5108.072360] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5108.072367] ? apic_timer_interrupt+0xf/0x20 [ 5108.072372] [ 5108.072375] ? native_safe_halt+0x2e/0x50 [ 5108.072380] kvm_wait+0xcd/0xe0 [ 5108.072388] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5108.072395] do_raw_spin_lock+0x177/0x190 [ 5108.072400] _raw_spin_lock+0x21/0x30 [ 5108.072405] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5108.072711] ? do_raw_spin_unlock+0x75/0x190 [ 5108.072717] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5108.072942] ? do_wait_intr+0xf0/0xf0 [ 5108.072951] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5108.073188] mdc_reint+0x2d/0x110 [mdc] [ 5108.073233] mdc_unlink+0x658/0x870 [mdc] [ 5108.073263] lmv_unlink+0x34d/0xbd0 [lmv] [ 5108.073298] ll_unlink+0x24d/0x7f0 [lustre] [ 5108.073427] vfs_unlink+0x202/0x330 [ 5108.073435] do_unlinkat+0x2fa/0x460 [ 5108.073441] __x64_sys_unlink+0x37/0x50 [ 5108.073444] do_syscall_64+0xc1/0x440 [ 5108.073453] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5108.073459] RIP: 0033:0x7fe02c11910b [ 5108.073467] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5108.073471] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5108.073476] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5108.073479] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5108.073481] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5108.073483] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5108.073485] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5108.437684] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5108.449710] ? __next_timer_interrupt+0x160/0x160 [ 5108.454122] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5108.462643] ? do_wait_intr+0xf0/0xf0 [ 5108.465756] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5108.467654] kthread+0x1d1/0x200 [ 5108.469204] ? set_kthread_struct+0x70/0x70 [ 5108.474150] ret_from_fork+0x1f/0x30 [ 5136.060069] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 5136.063315] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 5136.063882] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5136.063956] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5136.063962] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5136.063965] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5136.063978] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5136.064167] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5136.064172] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5136.064174] RDX: 0000000000007d0a RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5136.064177] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5136.064179] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5136.064182] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5136.064185] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5136.064188] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5136.064190] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5136.064198] Call Trace: [ 5136.064204] [ 5136.064210] ? show_regs.cold.9+0x22/0x2f [ 5136.064220] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5136.064226] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5136.064230] ? watchdog+0x40/0x40 [ 5136.064234] ? hrtimer_interrupt+0x128/0x2c0 [ 5136.064238] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5136.064246] ? apic_timer_interrupt+0xf/0x20 [ 5136.064251] [ 5136.064254] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5136.064259] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5136.064264] do_raw_spin_lock+0x177/0x190 [ 5136.064269] _raw_spin_lock+0x21/0x30 [ 5136.064276] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5136.064555] obd_mod_free+0x46/0x140 [ptlrpc] [ 5136.065202] mdc_commit_open+0x91/0xb0 [mdc] [ 5136.065241] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5136.065445] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5136.065805] after_reply+0xe2c/0x1780 [ptlrpc] [ 5136.066037] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5136.066350] ? __next_timer_interrupt+0x160/0x160 [ 5136.066360] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5136.066596] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 5136.066610] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) [ 5136.066637] ? do_wait_intr+0xf0/0xf0 [ 5136.066644] mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5136.066762] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5136.066768] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5136.066666] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5136.066777] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5136.068468] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5136.068485] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5136.068489] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5136.068493] RAX: 0000000000006453 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5136.068496] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5136.068498] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5136.068500] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5136.068503] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5136.068506] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5136.068509] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5136.068512] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5136.068953] Call Trace: [ 5136.068961] [ 5136.068968] ? show_regs.cold.9+0x22/0x2f [ 5136.068980] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5136.068988] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5136.068993] ? watchdog+0x40/0x40 [ 5136.068998] ? hrtimer_interrupt+0x128/0x2c0 [ 5136.069005] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5136.069017] ? apic_timer_interrupt+0xf/0x20 [ 5136.069022] [ 5136.069025] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5136.069031] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5136.069037] do_raw_spin_lock+0x177/0x190 [ 5136.069042] _raw_spin_lock+0x21/0x30 [ 5136.069047] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5136.069319] process_one_work+0x2c8/0x700 [ 5136.069327] worker_thread+0x296/0x6e0 [ 5136.069331] ? rescuer_thread+0x570/0x570 [ 5136.069335] kthread+0x1d1/0x200 [ 5136.069340] ? set_kthread_struct+0x70/0x70 [ 5136.069344] ret_from_fork+0x1f/0x30 [ 5136.073781] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5136.073981] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5136.073989] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5136.073992] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5136.110110] kthread+0x1d1/0x200 [ 5136.110130] ? set_kthread_struct+0x70/0x70 [ 5136.110135] ret_from_fork+0x1f/0x30 [ 5136.589873] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5136.611510] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5136.614346] RAX: 0000000000005e09 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5136.619804] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5136.628564] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5136.634363] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5136.637573] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5136.641353] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5136.646669] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5136.649484] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5136.652482] Call Trace: [ 5136.653697] [ 5136.654509] ? show_regs.cold.9+0x22/0x2f [ 5136.656543] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5136.658353] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5136.660128] ? watchdog+0x40/0x40 [ 5136.661615] ? hrtimer_interrupt+0x128/0x2c0 [ 5136.664415] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5136.666894] ? apic_timer_interrupt+0xf/0x20 [ 5136.668633] [ 5136.669455] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5136.671618] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5136.673837] do_raw_spin_lock+0x177/0x190 [ 5136.675979] _raw_spin_lock+0x21/0x30 [ 5136.677509] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5136.683762] ? do_raw_spin_unlock+0x75/0x190 [ 5136.689892] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5136.695524] ? do_wait_intr+0xf0/0xf0 [ 5136.698966] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5136.702821] mdc_reint+0x2d/0x110 [mdc] [ 5136.708012] mdc_unlink+0x658/0x870 [mdc] [ 5136.713444] lmv_unlink+0x34d/0xbd0 [lmv] [ 5136.719291] ll_unlink+0x24d/0x7f0 [lustre] [ 5136.722997] vfs_unlink+0x202/0x330 [ 5136.724620] do_unlinkat+0x2fa/0x460 [ 5136.731006] __x64_sys_unlink+0x37/0x50 [ 5136.735048] do_syscall_64+0xc1/0x440 [ 5136.736327] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5136.742663] RIP: 0033:0x7fe02c11910b [ 5136.749906] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5136.768486] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5136.778994] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5136.787122] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5136.790301] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5136.793508] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5136.801437] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5164.059818] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5164.064066] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 5164.065204] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5164.065212] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5164.065290] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5164.065296] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5164.065299] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5164.065313] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5164.065318] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5164.065322] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5164.065325] RDX: 00000000000003bd RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5164.065327] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5164.065330] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5164.065333] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5164.065336] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5164.065339] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5164.065341] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5164.065351] Call Trace: [ 5164.066959] [ 5164.067125] ? show_regs.cold.9+0x22/0x2f [ 5164.067137] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5164.067144] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5164.067148] ? watchdog+0x40/0x40 [ 5164.067152] ? hrtimer_interrupt+0x128/0x2c0 [ 5164.067156] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5164.067177] ? apic_timer_interrupt+0xf/0x20 [ 5164.067183] [ 5164.067186] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5164.067192] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5164.067197] do_raw_spin_lock+0x177/0x190 [ 5164.067202] _raw_spin_lock+0x21/0x30 [ 5164.067213] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5164.069066] obd_mod_free+0x46/0x140 [ptlrpc] [ 5164.069387] mdc_commit_open+0x91/0xb0 [mdc] [ 5164.069422] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5164.070207] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5164.070400] after_reply+0xe2c/0x1780 [ptlrpc] [ 5164.071121] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5164.071690] ? __next_timer_interrupt+0x160/0x160 [ 5164.071700] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5164.072450] ? do_wait_intr+0xf0/0xf0 [ 5164.072460] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5164.072815] kthread+0x1d1/0x200 [ 5164.072821] ? set_kthread_struct+0x70/0x70 [ 5164.072825] ret_from_fork+0x1f/0x30 [ 5164.077394] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) [ 5164.083858] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5164.084171] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5164.084178] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5164.084185] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5164.089912] lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5164.090174] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5164.090181] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5164.090183] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5164.090201] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5164.090206] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5164.090210] RAX: 00000000000022ca RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5164.090212] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5164.090214] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5164.090216] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5164.090219] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5164.090222] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5164.090427] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5164.090432] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5164.090446] Call Trace: [ 5164.090452] [ 5164.090458] ? show_regs.cold.9+0x22/0x2f [ 5164.090473] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5164.090480] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5164.090488] ? watchdog+0x40/0x40 [ 5164.090491] ? hrtimer_interrupt+0x128/0x2c0 [ 5164.090495] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5164.090505] ? apic_timer_interrupt+0xf/0x20 [ 5164.090510] [ 5164.090513] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5164.090517] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5164.090521] do_raw_spin_lock+0x177/0x190 [ 5164.090526] _raw_spin_lock+0x21/0x30 [ 5164.090531] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5164.090954] ? do_raw_spin_unlock+0x75/0x190 [ 5164.090960] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5164.091640] ? do_wait_intr+0xf0/0xf0 [ 5164.091649] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5164.091863] mdc_reint+0x2d/0x110 [mdc] [ 5164.092125] mdc_unlink+0x658/0x870 [mdc] [ 5164.092162] lmv_unlink+0x34d/0xbd0 [lmv] [ 5164.092200] ll_unlink+0x24d/0x7f0 [lustre] [ 5164.092316] vfs_unlink+0x202/0x330 [ 5164.092329] do_unlinkat+0x2fa/0x460 [ 5164.092334] __x64_sys_unlink+0x37/0x50 [ 5164.092337] do_syscall_64+0xc1/0x440 [ 5164.092701] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5164.092708] RIP: 0033:0x7fe02c11910b [ 5164.092716] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5164.092720] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5164.092725] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5164.092727] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5164.092729] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5164.092731] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5164.092733] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5164.643881] [ 5164.644687] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5164.648113] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5164.662430] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5164.666500] RAX: 0000000000006796 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5164.671324] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5164.675023] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5164.679202] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5164.686035] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5164.690967] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5164.700214] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5164.709191] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5164.718973] Call Trace: [ 5164.727634] [ 5164.728374] ? show_regs.cold.9+0x22/0x2f [ 5164.735171] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5164.738071] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5164.744588] ? watchdog+0x40/0x40 [ 5164.745673] ? hrtimer_interrupt+0x128/0x2c0 [ 5164.755825] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5164.759756] ? apic_timer_interrupt+0xf/0x20 [ 5164.763226] [ 5164.766975] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5164.770696] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5164.773983] do_raw_spin_lock+0x177/0x190 [ 5164.776113] _raw_spin_lock+0x21/0x30 [ 5164.778207] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5164.781753] process_one_work+0x2c8/0x700 [ 5164.783913] worker_thread+0x296/0x6e0 [ 5164.788543] ? rescuer_thread+0x570/0x570 [ 5164.789951] kthread+0x1d1/0x200 [ 5164.791002] ? set_kthread_struct+0x70/0x70 [ 5164.793721] ret_from_fork+0x1f/0x30 [ 5192.057500] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5192.060017] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5192.063718] Modules linked in: lustre(O) [ 5192.064021] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 5192.064027] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5192.064150] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5192.064156] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5192.064162] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5192.064449] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5192.064473] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5192.064477] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5192.064481] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5192.064483] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5192.064486] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5192.064488] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5192.064490] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5192.064493] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5192.064496] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5192.064499] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5192.064507] Call Trace: [ 5192.064524] [ 5192.064530] ? show_regs.cold.9+0x22/0x2f [ 5192.064671] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5192.064689] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5192.064693] ? watchdog+0x40/0x40 [ 5192.064697] ? hrtimer_interrupt+0x128/0x2c0 [ 5192.064701] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5192.064709] ? apic_timer_interrupt+0xf/0x20 [ 5192.064713] [ 5192.064716] ? native_safe_halt+0x2e/0x50 [ 5192.064721] kvm_wait+0xcd/0xe0 [ 5192.064729] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5192.064736] do_raw_spin_lock+0x177/0x190 [ 5192.064740] _raw_spin_lock+0x21/0x30 [ 5192.064744] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5192.064960] process_one_work+0x2c8/0x700 [ 5192.064967] worker_thread+0x296/0x6e0 [ 5192.064972] ? rescuer_thread+0x570/0x570 [ 5192.064975] kthread+0x1d1/0x200 [ 5192.064980] ? set_kthread_struct+0x70/0x70 [ 5192.064983] ret_from_fork+0x1f/0x30 [ 5192.068429] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5192.068533] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5192.068539] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5192.068542] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5192.068554] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5192.068559] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5192.068563] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5192.068565] RDX: 00000000000040a8 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5192.068567] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5192.068570] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5192.068572] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5192.068574] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5192.068577] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5192.068579] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5192.068587] Call Trace: [ 5192.068593] [ 5192.068598] ? show_regs.cold.9+0x22/0x2f [ 5192.068812] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5192.068820] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5192.068825] ? watchdog+0x40/0x40 [ 5192.068829] ? hrtimer_interrupt+0x128/0x2c0 [ 5192.068834] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5192.068849] ? apic_timer_interrupt+0xf/0x20 [ 5192.068856] [ 5192.068860] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5192.068869] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5192.068874] do_raw_spin_lock+0x177/0x190 [ 5192.068882] _raw_spin_lock+0x21/0x30 [ 5192.068888] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5192.075724] osp(O) [ 5192.079423] obd_mod_free+0x46/0x140 [ptlrpc] [ 5192.115992] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5192.116083] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5192.116092] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5192.116094] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5192.116107] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5192.116111] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5192.116120] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5192.116122] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 5192.116124] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5192.116126] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5192.116129] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5192.116132] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5192.116135] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5192.116138] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5192.116146] Call Trace: [ 5192.116152] [ 5192.116157] ? show_regs.cold.9+0x22/0x2f [ 5192.116168] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5192.116176] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5192.116180] ? watchdog+0x40/0x40 [ 5192.116184] ? hrtimer_interrupt+0x128/0x2c0 [ 5192.116188] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5192.116202] ? apic_timer_interrupt+0xf/0x20 [ 5192.116206] [ 5192.116210] ? native_safe_halt+0x2e/0x50 [ 5192.116213] kvm_wait+0xcd/0xe0 [ 5192.116224] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5192.116231] do_raw_spin_lock+0x177/0x190 [ 5192.116235] _raw_spin_lock+0x21/0x30 [ 5192.116241] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5192.125855] mdc_commit_open+0x91/0xb0 [mdc] [ 5192.125906] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5192.126200] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5192.126514] after_reply+0xe2c/0x1780 [ptlrpc] [ 5192.126759] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5192.134587] ? do_raw_spin_unlock+0x75/0x190 [ 5192.134607] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5192.134911] ? do_wait_intr+0xf0/0xf0 [ 5192.134918] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5192.135134] mdc_reint+0x2d/0x110 [mdc] [ 5192.135173] mdc_unlink+0x658/0x870 [mdc] [ 5192.135203] lmv_unlink+0x34d/0xbd0 [lmv] [ 5192.135236] ll_unlink+0x24d/0x7f0 [lustre] [ 5192.135573] vfs_unlink+0x202/0x330 [ 5192.135583] do_unlinkat+0x2fa/0x460 [ 5192.135589] __x64_sys_unlink+0x37/0x50 [ 5192.135593] do_syscall_64+0xc1/0x440 [ 5192.135601] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5192.135610] RIP: 0033:0x7fe02c11910b [ 5192.135618] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5192.135622] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5192.135627] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5192.135630] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5192.135632] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5192.135634] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5192.135637] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5192.686070] ? __next_timer_interrupt+0x160/0x160 [ 5192.687434] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5192.692282] ? do_wait_intr+0xf0/0xf0 [ 5192.694214] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5192.702588] kthread+0x1d1/0x200 [ 5192.705774] ? set_kthread_struct+0x70/0x70 [ 5192.710374] ret_from_fork+0x1f/0x30 [ 5220.060068] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5220.060454] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5220.074162] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 5220.074176] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5220.074256] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5220.074261] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5220.074267] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5220.074558] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5220.075269] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix [ 5220.074568] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5220.075465] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5220.075470] RAX: 0000000000005137 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5220.075473] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5220.075474] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5220.075477] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5220.075479] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5220.075482] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5220.075485] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5220.075487] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5220.075496] Call Trace: [ 5220.075502] [ 5220.075508] ? show_regs.cold.9+0x22/0x2f [ 5220.075520] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5220.075527] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5220.075530] ? watchdog+0x40/0x40 [ 5220.075535] ? hrtimer_interrupt+0x128/0x2c0 [ 5220.075538] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5220.075548] ? apic_timer_interrupt+0xf/0x20 [ 5220.075553] [ 5220.075555] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5220.077356] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5220.077404] serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5220.077418] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5220.077424] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5220.077427] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5220.077439] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5220.077442] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5220.077447] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5220.077450] RDX: 0000000000005e94 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5220.077452] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5220.077454] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5220.077460] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5220.077463] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5220.077468] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5220.077470] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5220.077481] Call Trace: [ 5220.077491] [ 5220.077494] ? show_regs.cold.9+0x22/0x2f [ 5220.077509] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5220.077514] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5220.077521] ? watchdog+0x40/0x40 [ 5220.077529] ? hrtimer_interrupt+0x128/0x2c0 [ 5220.077537] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5220.077548] ? apic_timer_interrupt+0xf/0x20 [ 5220.077556] [ 5220.077559] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5220.077575] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5220.077583] do_raw_spin_lock+0x177/0x190 [ 5220.077588] _raw_spin_lock+0x21/0x30 [ 5220.077595] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5220.077861] obd_mod_free+0x46/0x140 [ptlrpc] [ 5220.078056] mdc_commit_open+0x91/0xb0 [mdc] [ 5220.078089] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5220.078295] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5220.078495] after_reply+0xe2c/0x1780 [ptlrpc] [ 5220.078696] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5220.078893] ? __next_timer_interrupt+0x160/0x160 [ 5220.078903] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5220.080148] ? do_wait_intr+0xf0/0xf0 [ 5220.080263] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5220.080908] kthread+0x1d1/0x200 [ 5220.080914] ? set_kthread_struct+0x70/0x70 [ 5220.080918] ret_from_fork+0x1f/0x30 [ 5220.082589] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5220.082666] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5220.082672] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5220.082677] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5220.082687] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5220.082692] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5220.082697] RAX: 0000000000003a5b RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5220.082699] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5220.082703] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5220.082705] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5220.082707] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5220.082710] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5220.082712] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5220.082714] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5220.082722] Call Trace: [ 5220.082727] [ 5220.082732] ? show_regs.cold.9+0x22/0x2f [ 5220.082754] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5220.082772] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5220.082777] ? watchdog+0x40/0x40 [ 5220.082781] ? hrtimer_interrupt+0x128/0x2c0 [ 5220.082785] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5220.082792] ? apic_timer_interrupt+0xf/0x20 [ 5220.082798] [ 5220.082800] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5220.082804] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5220.082808] do_raw_spin_lock+0x177/0x190 [ 5220.082812] _raw_spin_lock+0x21/0x30 [ 5220.082817] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5220.083093] ? do_raw_spin_unlock+0x75/0x190 [ 5220.083099] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5220.083333] ? do_wait_intr+0xf0/0xf0 [ 5220.083341] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5220.093720] do_raw_spin_lock+0x177/0x190 [ 5220.093753] _raw_spin_lock+0x21/0x30 [ 5220.093760] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5220.161429] mdc_reint+0x2d/0x110 [mdc] [ 5220.161489] mdc_unlink+0x658/0x870 [mdc] [ 5220.161517] lmv_unlink+0x34d/0xbd0 [lmv] [ 5220.161552] ll_unlink+0x24d/0x7f0 [lustre] [ 5220.162776] vfs_unlink+0x202/0x330 [ 5220.162789] do_unlinkat+0x2fa/0x460 [ 5220.162795] __x64_sys_unlink+0x37/0x50 [ 5220.162799] do_syscall_64+0xc1/0x440 [ 5220.162807] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5220.162890] RIP: 0033:0x7fe02c11910b [ 5220.162901] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5220.162906] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5220.162911] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5220.162913] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5220.162916] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5220.162918] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5220.162920] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5221.039274] process_one_work+0x2c8/0x700 [ 5221.041195] worker_thread+0x296/0x6e0 [ 5221.050062] ? rescuer_thread+0x570/0x570 [ 5221.051754] kthread+0x1d1/0x200 [ 5221.056883] ? set_kthread_struct+0x70/0x70 [ 5221.059321] ret_from_fork+0x1f/0x30 [ 5248.060034] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5248.061205] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5248.063307] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5248.063384] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5248.063392] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5248.063395] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5248.063407] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5248.063411] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5248.063416] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5248.063418] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5248.063421] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5248.063423] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5248.063425] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5248.063428] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5248.063431] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5248.063434] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5248.063442] Call Trace: [ 5248.063447] [ 5248.063453] ? show_regs.cold.9+0x22/0x2f [ 5248.063464] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5248.063471] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5248.063475] ? watchdog+0x40/0x40 [ 5248.063479] ? hrtimer_interrupt+0x128/0x2c0 [ 5248.063484] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5248.063491] ? apic_timer_interrupt+0xf/0x20 [ 5248.063495] [ 5248.063498] ? native_safe_halt+0x2e/0x50 [ 5248.063502] kvm_wait+0xcd/0xe0 [ 5248.063511] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5248.063519] do_raw_spin_lock+0x177/0x190 [ 5248.063524] _raw_spin_lock+0x21/0x30 [ 5248.063530] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5248.064494] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 5248.064508] Modules linked in: [ 5248.064518] obd_mod_free+0x46/0x140 [ptlrpc] [ 5248.064733] mdc_commit_open+0x91/0xb0 [mdc] [ 5248.064778] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5248.065218] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5248.065415] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5248.065422] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5248.065430] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5248.066571] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5248.066591] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5248.066595] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5248.067433] RAX: 0000000000004988 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5248.067436] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5248.067439] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5248.067442] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5248.067444] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5248.067449] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5248.067452] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5248.067455] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5248.067464] Call Trace: [ 5248.067471] [ 5248.067478] ? show_regs.cold.9+0x22/0x2f [ 5248.067494] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5248.067501] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5248.067506] ? watchdog+0x40/0x40 [ 5248.067510] ? hrtimer_interrupt+0x128/0x2c0 [ 5248.067515] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5248.067525] ? apic_timer_interrupt+0xf/0x20 [ 5248.067530] [ 5248.067533] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5248.067539] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5248.067545] do_raw_spin_lock+0x177/0x190 [ 5248.067550] _raw_spin_lock+0x21/0x30 [ 5248.067555] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5248.068066] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5248.068407] after_reply+0xe2c/0x1780 [ptlrpc] [ 5248.069396] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5248.070640] ? __next_timer_interrupt+0x160/0x160 [ 5248.070652] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5248.070906] ? do_wait_intr+0xf0/0xf0 [ 5248.070916] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5248.071843] kthread+0x1d1/0x200 [ 5248.071852] ? set_kthread_struct+0x70/0x70 [ 5248.071856] ret_from_fork+0x1f/0x30 [ 5248.074278] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5248.074465] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5248.074471] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5248.074474] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5248.074485] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5248.074489] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5248.074495] RAX: 0000000000006f86 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5248.074498] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5248.074501] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5248.074503] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5248.074506] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5248.074510] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5248.074513] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5248.074516] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5248.074524] Call Trace: [ 5248.074530] [ 5248.074536] ? show_regs.cold.9+0x22/0x2f [ 5248.074546] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5248.074553] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5248.074558] ? watchdog+0x40/0x40 [ 5248.074562] ? hrtimer_interrupt+0x128/0x2c0 [ 5248.074575] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5248.074584] ? apic_timer_interrupt+0xf/0x20 [ 5248.074589] [ 5248.074591] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5248.074596] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5248.074601] do_raw_spin_lock+0x177/0x190 [ 5248.074605] _raw_spin_lock+0x21/0x30 [ 5248.074611] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5248.074904] ? do_raw_spin_unlock+0x75/0x190 [ 5248.074909] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5248.075429] ? do_wait_intr+0xf0/0xf0 [ 5248.075437] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5248.076147] mdc_reint+0x2d/0x110 [mdc] [ 5248.076187] mdc_unlink+0x658/0x870 [mdc] [ 5248.076217] lmv_unlink+0x34d/0xbd0 [lmv] [ 5248.076247] ll_unlink+0x24d/0x7f0 [lustre] [ 5248.076359] vfs_unlink+0x202/0x330 [ 5248.076369] do_unlinkat+0x2fa/0x460 [ 5248.076373] __x64_sys_unlink+0x37/0x50 [ 5248.076377] do_syscall_64+0xc1/0x440 [ 5248.076383] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5248.076390] RIP: 0033:0x7fe02c11910b [ 5248.076396] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5248.076400] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5248.076406] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5248.076409] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5248.076410] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5248.076413] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5248.076415] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5248.676793] process_one_work+0x2c8/0x700 [ 5248.678355] worker_thread+0x296/0x6e0 [ 5248.679745] ? rescuer_thread+0x570/0x570 [ 5248.681345] kthread+0x1d1/0x200 [ 5248.682784] ? set_kthread_struct+0x70/0x70 [ 5248.684641] ret_from_fork+0x1f/0x30 [ 5262.168323] rcu: INFO: rcu_sched self-detected stall on CPU [ 5262.175805] rcu: 1-....: (2099607 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=602716 [ 5262.189896] (t=3120076 jiffies g=278549 q=16094) [ 5262.198177] Sending NMI from CPU 1 to CPUs 0: [ 5262.202950] NMI backtrace for cpu 0 [ 5262.202956] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5262.202958] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5262.202959] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5262.202961] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5262.202963] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 [ 5262.202966] RAX: 000000000000634f RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5262.202967] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5262.202969] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5262.202970] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5262.202971] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5262.202973] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5262.202974] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5262.202975] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5262.202976] Call Trace: [ 5262.202977] [ 5262.202978] ? show_regs.cold.9+0x22/0x2f [ 5262.202979] ? nmi_cpu_backtrace.cold.8+0x98/0xa2 [ 5262.202981] ? nmi_cpu_backtrace_handler+0x15/0x40 [ 5262.202982] ? nmi_handle+0xbd/0x210 [ 5262.202983] ? default_do_nmi+0x4b/0x1b0 [ 5262.202983] ? do_nmi+0x280/0x380 [ 5262.202984] ? end_repeat_nmi+0x16/0x1f [ 5262.202985] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5262.202986] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5262.202987] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5262.202988] [ 5262.202989] do_raw_spin_lock+0x177/0x190 [ 5262.202990] _raw_spin_lock+0x21/0x30 [ 5262.202991] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5262.202992] ? do_raw_spin_unlock+0x75/0x190 [ 5262.202993] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5262.202995] ? do_wait_intr+0xf0/0xf0 [ 5262.202996] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5262.202997] mdc_reint+0x2d/0x110 [mdc] [ 5262.202998] mdc_unlink+0x658/0x870 [mdc] [ 5262.202999] lmv_unlink+0x34d/0xbd0 [lmv] [ 5262.203001] ll_unlink+0x24d/0x7f0 [lustre] [ 5262.203002] vfs_unlink+0x202/0x330 [ 5262.203003] do_unlinkat+0x2fa/0x460 [ 5262.203004] __x64_sys_unlink+0x37/0x50 [ 5262.203005] do_syscall_64+0xc1/0x440 [ 5262.203006] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5262.203007] RIP: 0033:0x7fe02c11910b [ 5262.203008] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5262.203011] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5262.203013] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5262.203015] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5262.203016] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5262.203018] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5262.203020] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5262.205274] NMI backtrace for cpu 1 [ 5262.729137] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5262.735078] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5262.740140] Call Trace: [ 5262.742412] [ 5262.743254] ? dump_stack+0xbb/0x10e [ 5262.748783] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 5262.755157] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 5262.761282] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 5262.763797] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 5262.767074] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 5262.769367] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 5262.771585] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 5262.773828] ? update_process_times+0x6d/0xc0 [ 5262.775771] ? tick_sched_handle.isra.18+0x27/0x90 [ 5262.777821] ? tick_sched_timer+0x53/0xb0 [ 5262.780244] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5262.782287] ? tick_sched_handle.isra.18+0x90/0x90 [ 5262.788966] ? hrtimer_interrupt+0x128/0x2c0 [ 5262.790760] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5262.793625] ? apic_timer_interrupt+0xf/0x20 [ 5262.795767] [ 5262.797231] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5262.799745] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5262.802462] ? do_raw_spin_lock+0x177/0x190 [ 5262.804475] ? _raw_spin_lock+0x21/0x30 [ 5262.806217] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5262.809263] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 5262.812161] ? mdc_commit_open+0x91/0xb0 [mdc] [ 5262.814393] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5262.817104] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5262.819771] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 5262.822212] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5262.824656] ? __next_timer_interrupt+0x160/0x160 [ 5262.826623] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5262.829221] ? do_wait_intr+0xf0/0xf0 [ 5262.830930] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5262.833450] ? kthread+0x1d1/0x200 [ 5262.835054] ? set_kthread_struct+0x70/0x70 [ 5262.836672] ? ret_from_fork+0x1f/0x30 [ 5276.065155] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 5276.081642] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5276.132269] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5276.167966] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5276.185929] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5276.199169] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5276.207754] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5276.229330] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5276.236662] RAX: 0000000000000c27 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5276.248296] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5276.263137] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5276.272112] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5276.280868] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5276.291447] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5276.297383] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5276.303578] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5276.309824] Call Trace: [ 5276.310640] [ 5276.311500] ? show_regs.cold.9+0x22/0x2f [ 5276.313390] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5276.318338] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5276.321021] ? watchdog+0x40/0x40 [ 5276.322895] ? hrtimer_interrupt+0x128/0x2c0 [ 5276.329268] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5276.332693] ? apic_timer_interrupt+0xf/0x20 [ 5276.336556] [ 5276.338655] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5276.343273] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5276.347581] do_raw_spin_lock+0x177/0x190 [ 5276.352458] _raw_spin_lock+0x21/0x30 [ 5276.354476] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5276.357175] process_one_work+0x2c8/0x700 [ 5276.359384] worker_thread+0x296/0x6e0 [ 5276.360775] ? rescuer_thread+0x570/0x570 [ 5276.362065] kthread+0x1d1/0x200 [ 5276.363009] ? set_kthread_struct+0x70/0x70 [ 5276.366109] ret_from_fork+0x1f/0x30 [ 5288.057534] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5288.060020] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5288.071587] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5288.071668] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5288.071676] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5288.071679] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5288.071693] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5288.071697] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5288.071701] RAX: 0000000000000111 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5288.071704] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5288.071706] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5288.071708] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5288.071711] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5288.071714] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5288.071718] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5288.071720] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5288.071728] Call Trace: [ 5288.071734] [ 5288.071740] ? show_regs.cold.9+0x22/0x2f [ 5288.071750] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5288.072237] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5288.072242] ? watchdog+0x40/0x40 [ 5288.072247] ? hrtimer_interrupt+0x128/0x2c0 [ 5288.072251] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5288.072261] ? apic_timer_interrupt+0xf/0x20 [ 5288.072328] [ 5288.072333] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5288.072339] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5288.072344] do_raw_spin_lock+0x177/0x190 [ 5288.072349] _raw_spin_lock+0x21/0x30 [ 5288.072355] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5288.074126] ? do_raw_spin_unlock+0x75/0x190 [ 5288.074135] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5288.074594] ? do_wait_intr+0xf0/0xf0 [ 5288.074602] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5288.085796] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5288.085876] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5288.085884] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5288.085887] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5288.085900] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5288.085905] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5288.085909] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5288.085911] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5288.085913] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5288.085915] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5288.085917] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5288.085920] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5288.085923] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5288.085926] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5288.085935] Call Trace: [ 5288.085940] [ 5288.085946] ? show_regs.cold.9+0x22/0x2f [ 5288.085956] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5288.085962] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5288.085967] ? watchdog+0x40/0x40 [ 5288.085972] ? hrtimer_interrupt+0x128/0x2c0 [ 5288.085976] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5288.085983] ? apic_timer_interrupt+0xf/0x20 [ 5288.085987] [ 5288.085990] ? native_safe_halt+0x2e/0x50 [ 5288.086008] kvm_wait+0xcd/0xe0 [ 5288.086016] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5288.086024] do_raw_spin_lock+0x177/0x190 [ 5288.086029] _raw_spin_lock+0x21/0x30 [ 5288.086034] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5288.086326] obd_mod_free+0x46/0x140 [ptlrpc] [ 5288.086908] mdc_commit_open+0x91/0xb0 [mdc] [ 5288.086944] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5288.087216] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5288.087661] after_reply+0xe2c/0x1780 [ptlrpc] [ 5288.087927] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5288.088181] ? __next_timer_interrupt+0x160/0x160 [ 5288.088191] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5288.126539] mdc_reint+0x2d/0x110 [mdc] [ 5288.126590] mdc_unlink+0x658/0x870 [mdc] [ 5288.126616] lmv_unlink+0x34d/0xbd0 [lmv] [ 5288.126645] ll_unlink+0x24d/0x7f0 [lustre] [ 5288.127279] vfs_unlink+0x202/0x330 [ 5288.127293] do_unlinkat+0x2fa/0x460 [ 5288.127298] __x64_sys_unlink+0x37/0x50 [ 5288.127301] do_syscall_64+0xc1/0x440 [ 5288.127309] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5288.127317] RIP: 0033:0x7fe02c11910b [ 5288.127325] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5288.127330] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5288.127336] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5288.127338] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5288.127340] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5288.127342] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5288.127345] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5288.548686] ? do_wait_intr+0xf0/0xf0 [ 5288.549850] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5288.553178] kthread+0x1d1/0x200 [ 5288.554941] ? set_kthread_struct+0x70/0x70 [ 5288.557282] ret_from_fork+0x1f/0x30 [ 5304.064211] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 5304.071000] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5304.108371] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5304.116143] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5304.120050] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5304.122940] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5304.124615] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5304.131989] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5304.136240] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5304.138935] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5304.141722] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5304.144554] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5304.147531] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5304.150196] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5304.153436] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5304.155073] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5304.158664] Call Trace: [ 5304.159695] [ 5304.161610] ? show_regs.cold.9+0x22/0x2f [ 5304.163368] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5304.166743] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5304.169089] ? watchdog+0x40/0x40 [ 5304.171535] ? hrtimer_interrupt+0x128/0x2c0 [ 5304.175216] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5304.177052] ? apic_timer_interrupt+0xf/0x20 [ 5304.179870] [ 5304.180819] ? native_safe_halt+0x2e/0x50 [ 5304.183847] kvm_wait+0xcd/0xe0 [ 5304.185088] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5304.188364] do_raw_spin_lock+0x177/0x190 [ 5304.190273] _raw_spin_lock+0x21/0x30 [ 5304.193309] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5304.197156] process_one_work+0x2c8/0x700 [ 5304.200302] worker_thread+0x296/0x6e0 [ 5304.201765] ? rescuer_thread+0x570/0x570 [ 5304.204872] kthread+0x1d1/0x200 [ 5304.206372] ? set_kthread_struct+0x70/0x70 [ 5304.208133] ret_from_fork+0x1f/0x30 [ 5316.057375] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5316.062529] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5316.065548] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5316.065627] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5316.065635] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5316.065638] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5316.065652] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5316.065657] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5316.065661] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5316.065663] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 5316.065665] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5316.065667] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5316.065670] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5316.065674] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5316.065677] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5316.065679] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5316.065688] Call Trace: [ 5316.065693] [ 5316.065699] ? show_regs.cold.9+0x22/0x2f [ 5316.065709] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5316.066211] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5316.066216] ? watchdog+0x40/0x40 [ 5316.066219] ? hrtimer_interrupt+0x128/0x2c0 [ 5316.066223] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5316.066229] ? apic_timer_interrupt+0xf/0x20 [ 5316.066234] [ 5316.066237] ? native_safe_halt+0x2e/0x50 [ 5316.066241] kvm_wait+0xcd/0xe0 [ 5316.066249] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5316.066257] do_raw_spin_lock+0x177/0x190 [ 5316.066262] _raw_spin_lock+0x21/0x30 [ 5316.066267] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5316.067472] ? do_raw_spin_unlock+0x75/0x190 [ 5316.067480] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5316.067704] ? do_wait_intr+0xf0/0xf0 [ 5316.072225] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5316.118569] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5316.118647] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5316.118654] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5316.118657] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5316.118670] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5316.118675] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5316.118680] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5316.118682] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5316.118684] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5316.118686] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5316.118689] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5316.118692] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5316.118694] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5316.118697] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5316.118705] Call Trace: [ 5316.118711] [ 5316.118716] ? show_regs.cold.9+0x22/0x2f [ 5316.118726] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5316.118732] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5316.118736] ? watchdog+0x40/0x40 [ 5316.118740] ? hrtimer_interrupt+0x128/0x2c0 [ 5316.118744] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5316.118751] ? apic_timer_interrupt+0xf/0x20 [ 5316.118756] [ 5316.118758] ? native_safe_halt+0x2e/0x50 [ 5316.118762] kvm_wait+0xcd/0xe0 [ 5316.118769] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5316.118777] do_raw_spin_lock+0x177/0x190 [ 5316.118781] _raw_spin_lock+0x21/0x30 [ 5316.118786] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5316.119041] obd_mod_free+0x46/0x140 [ptlrpc] [ 5316.119318] mdc_commit_open+0x91/0xb0 [mdc] [ 5316.128040] mdc_reint+0x2d/0x110 [mdc] [ 5316.128514] mdc_unlink+0x658/0x870 [mdc] [ 5316.128542] lmv_unlink+0x34d/0xbd0 [lmv] [ 5316.128575] ll_unlink+0x24d/0x7f0 [lustre] [ 5316.128701] vfs_unlink+0x202/0x330 [ 5316.128710] do_unlinkat+0x2fa/0x460 [ 5316.128716] __x64_sys_unlink+0x37/0x50 [ 5316.128720] do_syscall_64+0xc1/0x440 [ 5316.128728] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5316.128739] RIP: 0033:0x7fe02c11910b [ 5316.128747] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5316.128752] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5316.128757] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5316.128759] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5316.128761] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5316.128763] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5316.128765] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5316.701720] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5316.704818] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5316.706277] after_reply+0xe2c/0x1780 [ptlrpc] [ 5316.712573] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5316.715756] ? __next_timer_interrupt+0x160/0x160 [ 5316.723782] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5316.727058] ? do_wait_intr+0xf0/0xf0 [ 5316.732798] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5316.753096] kthread+0x1d1/0x200 [ 5316.755944] ? set_kthread_struct+0x70/0x70 [ 5316.766980] ret_from_fork+0x1f/0x30 [ 5332.067898] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 5332.075083] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5332.115388] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5332.124257] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5332.131511] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5332.139543] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5332.148543] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5332.164136] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5332.171088] RAX: 0000000000007f86 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5332.175386] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5332.180579] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5332.187503] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5332.191405] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5332.193323] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5332.197042] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5332.200390] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5332.204450] Call Trace: [ 5332.206202] [ 5332.206798] ? show_regs.cold.9+0x22/0x2f [ 5332.209034] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5332.211591] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5332.214327] ? watchdog+0x40/0x40 [ 5332.219538] ? hrtimer_interrupt+0x128/0x2c0 [ 5332.220671] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5332.221924] ? apic_timer_interrupt+0xf/0x20 [ 5332.222911] [ 5332.228489] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5332.233445] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5332.238396] do_raw_spin_lock+0x177/0x190 [ 5332.243337] _raw_spin_lock+0x21/0x30 [ 5332.245604] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5332.255304] process_one_work+0x2c8/0x700 [ 5332.259571] worker_thread+0x296/0x6e0 [ 5332.264531] ? rescuer_thread+0x570/0x570 [ 5332.267701] kthread+0x1d1/0x200 [ 5332.273083] ? set_kthread_struct+0x70/0x70 [ 5332.276714] ret_from_fork+0x1f/0x30 [ 5344.057045] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 5344.060458] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 5344.062646] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5344.062726] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5344.062732] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5344.062735] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5344.062747] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5344.062752] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5344.062756] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5344.062758] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 5344.062761] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5344.075902] Modules linked in: [ 5344.093184] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5344.093195] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5344.093199] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5344.093202] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5344.093204] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5344.093212] Call Trace: [ 5344.093219] [ 5344.093226] ? show_regs.cold.9+0x22/0x2f [ 5344.093238] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5344.093245] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5344.093249] ? watchdog+0x40/0x40 [ 5344.093254] ? hrtimer_interrupt+0x128/0x2c0 [ 5344.093257] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5344.093266] ? apic_timer_interrupt+0xf/0x20 [ 5344.093270] [ 5344.093273] ? native_safe_halt+0x2e/0x50 [ 5344.093277] kvm_wait+0xcd/0xe0 [ 5344.093285] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5344.093292] do_raw_spin_lock+0x177/0x190 [ 5344.093297] _raw_spin_lock+0x21/0x30 [ 5344.093304] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5344.094011] ? do_raw_spin_unlock+0x75/0x190 [ 5344.094029] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5344.094251] ? do_wait_intr+0xf0/0xf0 [ 5344.094259] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5344.094526] mdc_reint+0x2d/0x110 [mdc] [ 5344.094570] mdc_unlink+0x658/0x870 [mdc] [ 5344.094598] lmv_unlink+0x34d/0xbd0 [lmv] [ 5344.094630] ll_unlink+0x24d/0x7f0 [lustre] [ 5344.094758] vfs_unlink+0x202/0x330 [ 5344.094766] do_unlinkat+0x2fa/0x460 [ 5344.094770] __x64_sys_unlink+0x37/0x50 [ 5344.094774] do_syscall_64+0xc1/0x440 [ 5344.094780] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5344.094786] RIP: 0033:0x7fe02c11910b [ 5344.094794] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5344.094799] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5344.094804] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5344.094806] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5344.094808] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5344.094811] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5344.094813] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5344.244641] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5344.280814] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5344.284740] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5344.287400] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5344.288456] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5344.293976] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5344.296232] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5344.298102] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5344.300648] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5344.303717] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5344.306534] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5344.308451] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5344.312101] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5344.314639] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5344.316911] Call Trace: [ 5344.317695] [ 5344.318321] ? show_regs.cold.9+0x22/0x2f [ 5344.319486] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5344.325188] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5344.330276] ? watchdog+0x40/0x40 [ 5344.333159] ? hrtimer_interrupt+0x128/0x2c0 [ 5344.334606] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5344.339933] ? apic_timer_interrupt+0xf/0x20 [ 5344.345144] [ 5344.345988] ? native_safe_halt+0x2e/0x50 [ 5344.350750] kvm_wait+0xcd/0xe0 [ 5344.354526] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5344.359691] do_raw_spin_lock+0x177/0x190 [ 5344.364952] _raw_spin_lock+0x21/0x30 [ 5344.370097] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5344.375173] obd_mod_free+0x46/0x140 [ptlrpc] [ 5344.380305] mdc_commit_open+0x91/0xb0 [mdc] [ 5344.382822] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5344.392188] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5344.396035] after_reply+0xe2c/0x1780 [ptlrpc] [ 5344.398799] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5344.405134] ? __next_timer_interrupt+0x160/0x160 [ 5344.408474] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5344.415887] ? do_wait_intr+0xf0/0xf0 [ 5344.418552] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5344.420469] kthread+0x1d1/0x200 [ 5344.425897] ? set_kthread_struct+0x70/0x70 [ 5344.428808] ret_from_fork+0x1f/0x30 [ 5360.066301] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 5360.078919] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5360.125551] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5360.135627] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5360.139481] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5360.142659] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5360.147636] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5360.167695] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5360.174777] RAX: 0000000000007c9e RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5360.179392] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5360.188088] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5360.196418] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5360.202755] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5360.209352] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5360.214626] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5360.221866] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5360.226640] Call Trace: [ 5360.228743] [ 5360.229757] ? show_regs.cold.9+0x22/0x2f [ 5360.237003] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5360.241261] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5360.243029] ? watchdog+0x40/0x40 [ 5360.245097] ? hrtimer_interrupt+0x128/0x2c0 [ 5360.250380] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5360.253091] ? apic_timer_interrupt+0xf/0x20 [ 5360.254161] [ 5360.254576] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5360.261817] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5360.265507] do_raw_spin_lock+0x177/0x190 [ 5360.267721] _raw_spin_lock+0x21/0x30 [ 5360.269776] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5360.273543] process_one_work+0x2c8/0x700 [ 5360.279353] worker_thread+0x296/0x6e0 [ 5360.280942] ? rescuer_thread+0x570/0x570 [ 5360.283091] kthread+0x1d1/0x200 [ 5360.284667] ? set_kthread_struct+0x70/0x70 [ 5360.288966] ret_from_fork+0x1f/0x30 [ 5372.058080] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 5372.060023] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 5372.064104] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5372.064180] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5372.064187] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5372.064190] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5372.064204] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5372.064208] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5372.064212] RAX: 0000000000001d13 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5372.064215] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5372.064217] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5372.064219] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5372.064222] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5372.064225] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5372.064228] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5372.064231] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5372.064247] Call Trace: [ 5372.064253] [ 5372.064258] ? show_regs.cold.9+0x22/0x2f [ 5372.064268] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5372.064275] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5372.064279] ? watchdog+0x40/0x40 [ 5372.064284] ? hrtimer_interrupt+0x128/0x2c0 [ 5372.064289] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5372.064297] ? apic_timer_interrupt+0xf/0x20 [ 5372.064302] [ 5372.064306] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5372.064311] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5372.064316] do_raw_spin_lock+0x177/0x190 [ 5372.064321] _raw_spin_lock+0x21/0x30 [ 5372.064327] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5372.067643] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5372.067714] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5372.067720] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5372.067722] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5372.067735] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5372.067739] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5372.067743] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5372.067746] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5372.067748] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5372.067750] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5372.067752] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5372.067755] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5372.067758] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5372.067760] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5372.067768] Call Trace: [ 5372.067773] [ 5372.067777] ? show_regs.cold.9+0x22/0x2f [ 5372.067787] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5372.067793] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5372.067796] ? watchdog+0x40/0x40 [ 5372.067799] ? hrtimer_interrupt+0x128/0x2c0 [ 5372.067803] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5372.067809] ? apic_timer_interrupt+0xf/0x20 [ 5372.067813] [ 5372.067816] ? native_safe_halt+0x2e/0x50 [ 5372.067819] kvm_wait+0xcd/0xe0 [ 5372.067827] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5372.067834] do_raw_spin_lock+0x177/0x190 [ 5372.067838] _raw_spin_lock+0x21/0x30 [ 5372.067843] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5372.068257] obd_mod_free+0x46/0x140 [ptlrpc] [ 5372.068449] mdc_commit_open+0x91/0xb0 [mdc] [ 5372.068486] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5372.068689] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5372.068947] after_reply+0xe2c/0x1780 [ptlrpc] [ 5372.069142] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5372.069335] ? __next_timer_interrupt+0x160/0x160 [ 5372.069346] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5372.069561] ? do_wait_intr+0xf0/0xf0 [ 5372.069571] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5372.070133] kthread+0x1d1/0x200 [ 5372.070139] ? set_kthread_struct+0x70/0x70 [ 5372.070144] ret_from_fork+0x1f/0x30 [ 5372.538043] ? do_raw_spin_unlock+0x75/0x190 [ 5372.540773] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5372.543646] ? do_wait_intr+0xf0/0xf0 [ 5372.544996] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5372.548045] mdc_reint+0x2d/0x110 [mdc] [ 5372.549806] mdc_unlink+0x658/0x870 [mdc] [ 5372.551662] lmv_unlink+0x34d/0xbd0 [lmv] [ 5372.553973] ll_unlink+0x24d/0x7f0 [lustre] [ 5372.555882] vfs_unlink+0x202/0x330 [ 5372.557658] do_unlinkat+0x2fa/0x460 [ 5372.559335] __x64_sys_unlink+0x37/0x50 [ 5372.560861] do_syscall_64+0xc1/0x440 [ 5372.562675] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5372.564868] RIP: 0033:0x7fe02c11910b [ 5372.566502] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5372.574263] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5372.577343] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5372.585896] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5372.590513] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5372.595483] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5372.599066] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5388.065932] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 5388.075402] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5388.130617] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5388.148439] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5388.154276] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5388.167468] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5388.169232] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5388.206014] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5388.216601] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5388.225781] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5388.234906] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5388.241078] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5388.244028] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5388.246840] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5388.250768] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5388.253047] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5388.259972] Call Trace: [ 5388.261317] [ 5388.262216] ? show_regs.cold.9+0x22/0x2f [ 5388.268179] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5388.270619] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5388.274052] ? watchdog+0x40/0x40 [ 5388.275714] ? hrtimer_interrupt+0x128/0x2c0 [ 5388.277973] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5388.280496] ? apic_timer_interrupt+0xf/0x20 [ 5388.283545] [ 5388.284455] ? native_safe_halt+0x2e/0x50 [ 5388.287559] kvm_wait+0xcd/0xe0 [ 5388.288497] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5388.292471] do_raw_spin_lock+0x177/0x190 [ 5388.294197] _raw_spin_lock+0x21/0x30 [ 5388.295809] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5388.298426] process_one_work+0x2c8/0x700 [ 5388.300028] worker_thread+0x296/0x6e0 [ 5388.301398] ? rescuer_thread+0x570/0x570 [ 5388.302829] kthread+0x1d1/0x200 [ 5388.307925] ? set_kthread_struct+0x70/0x70 [ 5388.311878] ret_from_fork+0x1f/0x30 [ 5400.057773] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 5400.060099] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 5400.067278] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5400.067351] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5400.067359] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5400.067361] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5400.067374] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5400.067378] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5400.067382] RAX: 0000000000006a16 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5400.067384] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5400.067386] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5400.067388] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5400.067391] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5400.067394] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5400.067397] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5400.067400] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5400.067408] Call Trace: [ 5400.067414] [ 5400.067419] ? show_regs.cold.9+0x22/0x2f [ 5400.067440] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5400.067748] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5400.067753] ? watchdog+0x40/0x40 [ 5400.067757] ? hrtimer_interrupt+0x128/0x2c0 [ 5400.067762] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5400.067774] ? apic_timer_interrupt+0xf/0x20 [ 5400.067778] [ 5400.067782] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5400.067786] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5400.067791] do_raw_spin_lock+0x177/0x190 [ 5400.067796] _raw_spin_lock+0x21/0x30 [ 5400.067801] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5400.076587] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5400.076665] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5400.076674] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5400.076676] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5400.076691] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5400.076695] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5400.076699] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5400.076701] RDX: 0000000000007bd1 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5400.076703] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5400.076705] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5400.076708] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5400.076710] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5400.076713] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5400.076716] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5400.076725] Call Trace: [ 5400.076730] [ 5400.076735] ? show_regs.cold.9+0x22/0x2f [ 5400.076745] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5400.076752] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5400.076757] ? watchdog+0x40/0x40 [ 5400.076763] ? hrtimer_interrupt+0x128/0x2c0 [ 5400.076768] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5400.076777] ? apic_timer_interrupt+0xf/0x20 [ 5400.076781] [ 5400.076784] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5400.076789] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5400.076793] do_raw_spin_lock+0x177/0x190 [ 5400.076797] _raw_spin_lock+0x21/0x30 [ 5400.076803] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5400.077089] obd_mod_free+0x46/0x140 [ptlrpc] [ 5400.077300] mdc_commit_open+0x91/0xb0 [mdc] [ 5400.077491] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5400.077699] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5400.077912] after_reply+0xe2c/0x1780 [ptlrpc] [ 5400.078266] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5400.078614] ? __next_timer_interrupt+0x160/0x160 [ 5400.078624] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5400.151330] ? do_raw_spin_unlock+0x75/0x190 [ 5400.151353] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5400.151626] ? do_wait_intr+0xf0/0xf0 [ 5400.151634] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5400.156323] ? do_wait_intr+0xf0/0xf0 [ 5400.156343] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5400.156987] kthread+0x1d1/0x200 [ 5400.156994] ? set_kthread_struct+0x70/0x70 [ 5400.156998] ret_from_fork+0x1f/0x30 [ 5400.635990] mdc_reint+0x2d/0x110 [mdc] [ 5400.641980] mdc_unlink+0x658/0x870 [mdc] [ 5400.643630] lmv_unlink+0x34d/0xbd0 [lmv] [ 5400.645268] ll_unlink+0x24d/0x7f0 [lustre] [ 5400.646636] vfs_unlink+0x202/0x330 [ 5400.647647] do_unlinkat+0x2fa/0x460 [ 5400.648784] __x64_sys_unlink+0x37/0x50 [ 5400.650349] do_syscall_64+0xc1/0x440 [ 5400.651898] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5400.653707] RIP: 0033:0x7fe02c11910b [ 5400.655381] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5400.662268] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5400.665614] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5400.668796] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5400.672373] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5400.675553] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5400.680522] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5416.074048] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 5416.083077] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5416.134272] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5416.143813] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5416.151257] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5416.160479] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5416.169427] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5416.193671] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5416.203289] RAX: 000000000000760a RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5416.207329] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5416.216468] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5416.220837] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5416.227403] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5416.236673] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5416.243303] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5416.246938] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5416.257016] Call Trace: [ 5416.259894] [ 5416.260837] ? show_regs.cold.9+0x22/0x2f [ 5416.262602] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5416.267284] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5416.271659] ? watchdog+0x40/0x40 [ 5416.277288] ? hrtimer_interrupt+0x128/0x2c0 [ 5416.281605] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5416.285919] ? apic_timer_interrupt+0xf/0x20 [ 5416.288835] [ 5416.289991] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5416.294080] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5416.298303] do_raw_spin_lock+0x177/0x190 [ 5416.300918] _raw_spin_lock+0x21/0x30 [ 5416.306981] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5416.309891] process_one_work+0x2c8/0x700 [ 5416.311664] worker_thread+0x296/0x6e0 [ 5416.316014] ? rescuer_thread+0x570/0x570 [ 5416.321320] kthread+0x1d1/0x200 [ 5416.322784] ? set_kthread_struct+0x70/0x70 [ 5416.326744] ret_from_fork+0x1f/0x30 [ 5428.059849] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [writemany:36102] [ 5428.060030] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [ptlrpcd_rcv:2372] [ 5428.065559] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5428.065644] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5428.065651] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5428.065654] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5428.065669] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5428.065673] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5428.065678] RAX: 0000000000002395 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5428.065680] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5428.065683] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5428.065685] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5428.065687] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5428.065693] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5428.065697] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5428.065699] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5428.065707] Call Trace: [ 5428.065712] [ 5428.065717] ? show_regs.cold.9+0x22/0x2f [ 5428.065727] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5428.065738] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5428.065743] ? watchdog+0x40/0x40 [ 5428.065748] ? hrtimer_interrupt+0x128/0x2c0 [ 5428.065752] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5428.065760] ? apic_timer_interrupt+0xf/0x20 [ 5428.065765] [ 5428.065767] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5428.065772] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5428.065777] do_raw_spin_lock+0x177/0x190 [ 5428.065782] _raw_spin_lock+0x21/0x30 [ 5428.065788] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5428.066065] ? do_raw_spin_unlock+0x75/0x190 [ 5428.066071] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5428.066673] ? do_wait_intr+0xf0/0xf0 [ 5428.066681] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5428.066873] mdc_reint+0x2d/0x110 [mdc] [ 5428.066911] mdc_unlink+0x658/0x870 [mdc] [ 5428.066934] lmv_unlink+0x34d/0xbd0 [lmv] [ 5428.066965] ll_unlink+0x24d/0x7f0 [lustre] [ 5428.067595] vfs_unlink+0x202/0x330 [ 5428.067606] do_unlinkat+0x2fa/0x460 [ 5428.067611] __x64_sys_unlink+0x37/0x50 [ 5428.067615] do_syscall_64+0xc1/0x440 [ 5428.067621] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5428.067659] RIP: 0033:0x7fe02c11910b [ 5428.071470] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5428.071561] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5428.071569] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5428.071571] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5428.071584] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5428.071588] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5428.071592] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5428.071594] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5428.071596] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5428.071597] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5428.071600] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5428.071603] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5428.071605] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5428.071607] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5428.071615] Call Trace: [ 5428.071621] [ 5428.071625] ? show_regs.cold.9+0x22/0x2f [ 5428.071636] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5428.071642] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5428.071646] ? watchdog+0x40/0x40 [ 5428.071650] ? hrtimer_interrupt+0x128/0x2c0 [ 5428.071654] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5428.071660] ? apic_timer_interrupt+0xf/0x20 [ 5428.071665] [ 5428.071668] ? native_safe_halt+0x2e/0x50 [ 5428.071673] kvm_wait+0xcd/0xe0 [ 5428.071680] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5428.071685] do_raw_spin_lock+0x177/0x190 [ 5428.071690] _raw_spin_lock+0x21/0x30 [ 5428.071694] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5428.104831] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5428.104844] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5428.104851] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5428.104853] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5428.104855] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5428.104857] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5428.104860] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5428.450924] obd_mod_free+0x46/0x140 [ptlrpc] [ 5428.453230] mdc_commit_open+0x91/0xb0 [mdc] [ 5428.455118] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5428.457385] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5428.459894] after_reply+0xe2c/0x1780 [ptlrpc] [ 5428.461844] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5428.464205] ? __next_timer_interrupt+0x160/0x160 [ 5428.466225] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5428.468349] ? do_wait_intr+0xf0/0xf0 [ 5428.469941] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5428.472466] kthread+0x1d1/0x200 [ 5428.473888] ? set_kthread_struct+0x70/0x70 [ 5428.475622] ret_from_fork+0x1f/0x30 [ 5442.172143] rcu: INFO: rcu_sched self-detected stall on CPU [ 5442.174202] rcu: 1-....: (2208046 ticks this GP) idle=fe2/1/0x4000000000000002 softirq=133713/133713 fqs=635565 [ 5442.178457] (t=3300080 jiffies g=278549 q=16937) [ 5442.179863] Sending NMI from CPU 1 to CPUs 0: [ 5442.187354] NMI backtrace for cpu 0 [ 5442.187361] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5442.187363] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5442.187365] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5442.187367] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5442.187369] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000202 [ 5442.187371] RAX: 0000000000005a0b RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5442.187373] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5442.187374] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5442.187376] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5442.187377] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5442.187379] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5442.187461] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5442.187463] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5442.187465] Call Trace: [ 5442.187466] [ 5442.187466] ? show_regs.cold.9+0x22/0x2f [ 5442.187467] ? nmi_cpu_backtrace.cold.8+0x98/0xa2 [ 5442.187468] ? nmi_cpu_backtrace_handler+0x15/0x40 [ 5442.187470] ? nmi_handle+0xbd/0x210 [ 5442.187471] ? default_do_nmi+0x4b/0x1b0 [ 5442.187545] ? do_nmi+0x280/0x380 [ 5442.187546] ? end_repeat_nmi+0x16/0x1f [ 5442.187547] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5442.187548] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5442.187550] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5442.187551] [ 5442.187552] do_raw_spin_lock+0x177/0x190 [ 5442.187553] _raw_spin_lock+0x21/0x30 [ 5442.187554] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5442.187555] ? do_raw_spin_unlock+0x75/0x190 [ 5442.187556] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5442.187558] ? do_wait_intr+0xf0/0xf0 [ 5442.187559] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5442.187560] mdc_reint+0x2d/0x110 [mdc] [ 5442.187561] mdc_unlink+0x658/0x870 [mdc] [ 5442.187562] lmv_unlink+0x34d/0xbd0 [lmv] [ 5442.187563] ll_unlink+0x24d/0x7f0 [lustre] [ 5442.187564] vfs_unlink+0x202/0x330 [ 5442.187565] do_unlinkat+0x2fa/0x460 [ 5442.187566] __x64_sys_unlink+0x37/0x50 [ 5442.187567] do_syscall_64+0xc1/0x440 [ 5442.187568] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5442.187570] RIP: 0033:0x7fe02c11910b [ 5442.187572] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5442.187574] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5442.187577] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5442.187578] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5442.187580] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5442.187581] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5442.187583] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5442.187600] NMI backtrace for cpu 1 [ 5442.303643] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5442.315080] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5442.320885] Call Trace: [ 5442.322619] [ 5442.323253] ? dump_stack+0xbb/0x10e [ 5442.325512] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 5442.328952] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 5442.331944] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 5442.335188] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 5442.338655] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 5442.341824] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 5442.344889] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 5442.347554] ? update_process_times+0x6d/0xc0 [ 5442.349905] ? tick_sched_handle.isra.18+0x27/0x90 [ 5442.353552] ? tick_sched_timer+0x53/0xb0 [ 5442.355050] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5442.358329] ? tick_sched_handle.isra.18+0x90/0x90 [ 5442.359614] ? hrtimer_interrupt+0x128/0x2c0 [ 5442.362930] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5442.365310] ? apic_timer_interrupt+0xf/0x20 [ 5442.367463] [ 5442.369945] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5442.372764] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5442.375197] ? do_raw_spin_lock+0x177/0x190 [ 5442.376651] ? _raw_spin_lock+0x21/0x30 [ 5442.377966] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5442.379894] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 5442.381588] ? mdc_commit_open+0x91/0xb0 [mdc] [ 5442.383046] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5442.384803] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5442.386872] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 5442.388660] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5442.390357] ? __next_timer_interrupt+0x160/0x160 [ 5442.391878] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5442.393427] ? do_wait_intr+0xf0/0xf0 [ 5442.394607] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5442.396414] ? kthread+0x1d1/0x200 [ 5442.397565] ? set_kthread_struct+0x70/0x70 [ 5442.398901] ? ret_from_fork+0x1f/0x30 [ 5444.064041] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 5444.071403] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5444.106898] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5444.114329] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5444.123670] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5444.128126] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5444.130724] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5444.140362] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5444.144524] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5444.147496] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5444.149790] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5444.152288] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5444.154756] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5444.156860] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5444.159492] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5444.161553] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5444.164446] Call Trace: [ 5444.165314] [ 5444.165983] ? show_regs.cold.9+0x22/0x2f [ 5444.167754] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5444.169241] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5444.170845] ? watchdog+0x40/0x40 [ 5444.171779] ? hrtimer_interrupt+0x128/0x2c0 [ 5444.173061] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5444.174570] ? apic_timer_interrupt+0xf/0x20 [ 5444.176156] [ 5444.177006] ? native_safe_halt+0x2e/0x50 [ 5444.178399] kvm_wait+0xcd/0xe0 [ 5444.179240] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5444.180867] do_raw_spin_lock+0x177/0x190 [ 5444.182032] _raw_spin_lock+0x21/0x30 [ 5444.183184] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5444.185223] process_one_work+0x2c8/0x700 [ 5444.186829] worker_thread+0x296/0x6e0 [ 5444.187880] ? rescuer_thread+0x570/0x570 [ 5444.189119] kthread+0x1d1/0x200 [ 5444.190060] ? set_kthread_struct+0x70/0x70 [ 5444.192080] ret_from_fork+0x1f/0x30 [ 5468.057012] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5468.060017] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5468.060671] Modules linked in: [ 5468.064183] Modules linked in: [ 5468.065657] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5468.065740] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5468.065747] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5468.065750] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5468.065765] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5468.065770] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5468.065775] RAX: 00000000000055aa RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5468.065777] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5468.065784] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5468.065787] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5468.065790] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5468.065793] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5468.065795] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5468.065797] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5468.065804] Call Trace: [ 5468.065810] [ 5468.065815] ? show_regs.cold.9+0x22/0x2f [ 5468.065825] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5468.065830] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5468.065834] ? watchdog+0x40/0x40 [ 5468.065838] ? hrtimer_interrupt+0x128/0x2c0 [ 5468.065842] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5468.065862] ? apic_timer_interrupt+0xf/0x20 [ 5468.065867] [ 5468.065870] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5468.065875] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5468.065879] do_raw_spin_lock+0x177/0x190 [ 5468.065884] _raw_spin_lock+0x21/0x30 [ 5468.065889] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5468.066355] ? do_raw_spin_unlock+0x75/0x190 [ 5468.066361] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5468.066556] ? do_wait_intr+0xf0/0xf0 [ 5468.066564] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5468.066767] mdc_reint+0x2d/0x110 [mdc] [ 5468.066806] mdc_unlink+0x658/0x870 [mdc] [ 5468.066835] lmv_unlink+0x34d/0xbd0 [lmv] [ 5468.066879] ll_unlink+0x24d/0x7f0 [lustre] [ 5468.066997] vfs_unlink+0x202/0x330 [ 5468.067354] do_unlinkat+0x2fa/0x460 [ 5468.067360] __x64_sys_unlink+0x37/0x50 [ 5468.067365] do_syscall_64+0xc1/0x440 [ 5468.067372] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5468.067380] RIP: 0033:0x7fe02c11910b [ 5468.067388] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5468.067392] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5468.067397] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5468.067400] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5468.067402] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5468.067404] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5468.067407] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5468.289771] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5468.305670] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5468.308729] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5468.311205] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5468.313431] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5468.319043] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5468.322023] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5468.324805] RDX: 0000000000003229 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5468.328374] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5468.331338] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5468.334135] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5468.336897] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5468.340004] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5468.342337] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5468.344886] Call Trace: [ 5468.346578] [ 5468.347379] ? show_regs.cold.9+0x22/0x2f [ 5468.349293] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5468.351132] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5468.353100] ? watchdog+0x40/0x40 [ 5468.354453] ? hrtimer_interrupt+0x128/0x2c0 [ 5468.356622] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5468.358608] ? apic_timer_interrupt+0xf/0x20 [ 5468.360196] [ 5468.361063] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5468.363403] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5468.364971] do_raw_spin_lock+0x177/0x190 [ 5468.366808] _raw_spin_lock+0x21/0x30 [ 5468.368484] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5468.370558] obd_mod_free+0x46/0x140 [ptlrpc] [ 5468.372666] mdc_commit_open+0x91/0xb0 [mdc] [ 5468.374651] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5468.377126] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5468.379356] after_reply+0xe2c/0x1780 [ptlrpc] [ 5468.381360] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5468.383593] ? __next_timer_interrupt+0x160/0x160 [ 5468.385478] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5468.387457] ? do_wait_intr+0xf0/0xf0 [ 5468.389082] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5468.391577] kthread+0x1d1/0x200 [ 5468.392896] ? set_kthread_struct+0x70/0x70 [ 5468.394784] ret_from_fork+0x1f/0x30 [ 5472.064145] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 5472.069584] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5472.094666] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5472.100320] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5472.106033] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5472.109626] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5472.112715] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5472.123485] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5472.126398] RAX: 0000000000007742 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5472.130461] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5472.134093] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5472.137298] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5472.140657] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5472.144259] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5472.147831] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5472.150335] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5472.152929] Call Trace: [ 5472.153953] [ 5472.154803] ? show_regs.cold.9+0x22/0x2f [ 5472.156450] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5472.158328] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5472.160094] ? watchdog+0x40/0x40 [ 5472.161482] ? hrtimer_interrupt+0x128/0x2c0 [ 5472.163139] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5472.165321] ? apic_timer_interrupt+0xf/0x20 [ 5472.166886] [ 5472.167527] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5472.169658] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5472.171628] do_raw_spin_lock+0x177/0x190 [ 5472.173461] _raw_spin_lock+0x21/0x30 [ 5472.175237] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5472.178336] process_one_work+0x2c8/0x700 [ 5472.180528] worker_thread+0x296/0x6e0 [ 5472.182453] ? rescuer_thread+0x570/0x570 [ 5472.184463] kthread+0x1d1/0x200 [ 5472.185646] ? set_kthread_struct+0x70/0x70 [ 5472.187670] ret_from_fork+0x1f/0x30 [ 5496.057590] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5496.060066] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5496.067616] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5496.067689] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5496.067697] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5496.067700] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5496.067714] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5496.067717] RSP: 0018:ffffbad340e33b98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5496.067722] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5496.067724] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff993782034154 [ 5496.067726] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5496.067729] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5496.067731] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5496.067734] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5496.067737] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5496.067739] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5496.067748] Call Trace: [ 5496.067753] [ 5496.067758] ? show_regs.cold.9+0x22/0x2f [ 5496.067768] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5496.067774] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5496.067778] ? watchdog+0x40/0x40 [ 5496.067782] ? hrtimer_interrupt+0x128/0x2c0 [ 5496.067785] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5496.067792] ? apic_timer_interrupt+0xf/0x20 [ 5496.067797] [ 5496.067799] ? native_safe_halt+0x2e/0x50 [ 5496.067804] kvm_wait+0xcd/0xe0 [ 5496.067812] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5496.067819] do_raw_spin_lock+0x177/0x190 [ 5496.067824] _raw_spin_lock+0x21/0x30 [ 5496.067830] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5496.068114] ? do_raw_spin_unlock+0x75/0x190 [ 5496.068120] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5496.068322] ? do_wait_intr+0xf0/0xf0 [ 5496.068331] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5496.068542] mdc_reint+0x2d/0x110 [mdc] [ 5496.068582] mdc_unlink+0x658/0x870 [mdc] [ 5496.068610] lmv_unlink+0x34d/0xbd0 [lmv] [ 5496.068642] ll_unlink+0x24d/0x7f0 [lustre] [ 5496.068753] vfs_unlink+0x202/0x330 [ 5496.068761] do_unlinkat+0x2fa/0x460 [ 5496.068765] __x64_sys_unlink+0x37/0x50 [ 5496.068769] do_syscall_64+0xc1/0x440 [ 5496.068777] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5496.068783] RIP: 0033:0x7fe02c11910b [ 5496.068789] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5496.068792] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5496.068797] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5496.068799] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5496.068801] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5496.068803] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5496.068805] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5496.413409] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5496.457592] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5496.463516] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5496.468694] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5496.470723] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5496.489281] RSP: 0018:ffffbad3419cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5496.493674] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5496.497658] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5496.504687] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5496.512231] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5496.515520] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5496.524524] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5496.530264] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5496.537180] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5496.540564] Call Trace: [ 5496.541697] [ 5496.543259] ? show_regs.cold.9+0x22/0x2f [ 5496.546605] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5496.549487] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5496.553239] ? watchdog+0x40/0x40 [ 5496.554993] ? hrtimer_interrupt+0x128/0x2c0 [ 5496.564761] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5496.566435] ? apic_timer_interrupt+0xf/0x20 [ 5496.568961] [ 5496.569956] ? native_safe_halt+0x2e/0x50 [ 5496.572610] kvm_wait+0xcd/0xe0 [ 5496.574787] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5496.577933] do_raw_spin_lock+0x177/0x190 [ 5496.582231] _raw_spin_lock+0x21/0x30 [ 5496.586536] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5496.590240] obd_mod_free+0x46/0x140 [ptlrpc] [ 5496.592939] mdc_commit_open+0x91/0xb0 [mdc] [ 5496.595446] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5496.598366] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5496.608870] after_reply+0xe2c/0x1780 [ptlrpc] [ 5496.611549] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5496.615912] ? __next_timer_interrupt+0x160/0x160 [ 5496.618321] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5496.621959] ? do_wait_intr+0xf0/0xf0 [ 5496.624469] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5496.628565] kthread+0x1d1/0x200 [ 5496.630254] ? set_kthread_struct+0x70/0x70 [ 5496.632666] ret_from_fork+0x1f/0x30 [ 5500.064377] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:3:11403] [ 5500.069302] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5500.094157] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5500.102361] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5500.105876] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5500.110272] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5500.112059] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5500.119884] RSP: 0018:ffffbad341c33dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5500.123215] RAX: 0000000000000001 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5500.131335] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5500.133758] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5500.135802] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5500.137860] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5500.139829] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5500.142457] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5500.144011] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5500.146151] Call Trace: [ 5500.146964] [ 5500.147742] ? show_regs.cold.9+0x22/0x2f [ 5500.149023] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5500.150416] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5500.151991] ? watchdog+0x40/0x40 [ 5500.152903] ? hrtimer_interrupt+0x128/0x2c0 [ 5500.154115] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5500.155438] ? apic_timer_interrupt+0xf/0x20 [ 5500.156801] [ 5500.157417] ? native_safe_halt+0x2e/0x50 [ 5500.158772] kvm_wait+0xcd/0xe0 [ 5500.159775] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5500.161273] do_raw_spin_lock+0x177/0x190 [ 5500.162402] _raw_spin_lock+0x21/0x30 [ 5500.163492] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5500.165247] process_one_work+0x2c8/0x700 [ 5500.166396] worker_thread+0x296/0x6e0 [ 5500.167482] ? rescuer_thread+0x570/0x570 [ 5500.168623] kthread+0x1d1/0x200 [ 5500.169538] ? set_kthread_struct+0x70/0x70 [ 5500.170819] ret_from_fork+0x1f/0x30 [ 5524.057104] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5524.059910] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5524.060604] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5524.092323] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5524.092336] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5524.092338] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5524.092352] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5524.092356] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5524.092362] RAX: 0000000000002d18 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5524.092364] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5524.092366] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5524.092369] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5524.092371] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5524.092374] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5524.092377] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5524.092379] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5524.092386] Call Trace: [ 5524.092391] [ 5524.092396] ? show_regs.cold.9+0x22/0x2f [ 5524.092405] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5524.092412] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5524.092416] ? watchdog+0x40/0x40 [ 5524.092420] ? hrtimer_interrupt+0x128/0x2c0 [ 5524.092424] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5524.092433] ? apic_timer_interrupt+0xf/0x20 [ 5524.092438] [ 5524.092441] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5524.092445] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5524.092450] do_raw_spin_lock+0x177/0x190 [ 5524.092455] _raw_spin_lock+0x21/0x30 [ 5524.092462] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5524.092727] ? do_raw_spin_unlock+0x75/0x190 [ 5524.092732] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5524.106853] Modules linked in: [ 5524.112320] ? do_wait_intr+0xf0/0xf0 [ 5524.112345] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5524.112772] mdc_reint+0x2d/0x110 [mdc] [ 5524.112811] mdc_unlink+0x658/0x870 [mdc] [ 5524.112838] lmv_unlink+0x34d/0xbd0 [lmv] [ 5524.112883] ll_unlink+0x24d/0x7f0 [lustre] [ 5524.113011] vfs_unlink+0x202/0x330 [ 5524.113020] do_unlinkat+0x2fa/0x460 [ 5524.113024] __x64_sys_unlink+0x37/0x50 [ 5524.113027] do_syscall_64+0xc1/0x440 [ 5524.113034] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5524.113042] RIP: 0033:0x7fe02c11910b [ 5524.113048] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5524.113052] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5524.113057] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5524.113059] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5524.113061] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5524.113063] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5524.113065] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5524.310701] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5524.344191] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5524.351667] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5524.356450] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5524.359017] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5524.369971] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5524.375047] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5524.378238] RDX: 00000000000075ef RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5524.381909] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5524.385027] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5524.387944] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5524.390720] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5524.393855] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5524.395895] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5524.398730] Call Trace: [ 5524.400018] [ 5524.400786] ? show_regs.cold.9+0x22/0x2f [ 5524.402040] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5524.403779] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5524.410831] ? watchdog+0x40/0x40 [ 5524.412812] ? hrtimer_interrupt+0x128/0x2c0 [ 5524.414673] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5524.416800] ? apic_timer_interrupt+0xf/0x20 [ 5524.418669] [ 5524.419537] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5524.421560] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5524.423729] do_raw_spin_lock+0x177/0x190 [ 5524.425407] _raw_spin_lock+0x21/0x30 [ 5524.426981] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5524.429608] obd_mod_free+0x46/0x140 [ptlrpc] [ 5524.431904] mdc_commit_open+0x91/0xb0 [mdc] [ 5524.433640] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5524.435826] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5524.437834] after_reply+0xe2c/0x1780 [ptlrpc] [ 5524.439823] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5524.443810] ? __next_timer_interrupt+0x160/0x160 [ 5524.446968] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5524.453081] ? do_wait_intr+0xf0/0xf0 [ 5524.455021] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5524.457142] kthread+0x1d1/0x200 [ 5524.458210] ? set_kthread_struct+0x70/0x70 [ 5524.459399] ret_from_fork+0x1f/0x30 [ 5528.067407] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 5528.079730] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5528.126613] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5528.132772] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5528.139576] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5528.142342] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5528.145066] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5528.153856] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5528.159594] RAX: 0000000000000214 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5528.162328] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5528.167597] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5528.170982] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5528.174812] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5528.189563] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5528.193969] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5528.197104] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5528.200316] Call Trace: [ 5528.201556] [ 5528.202515] ? show_regs.cold.9+0x22/0x2f [ 5528.204585] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5528.207076] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5528.209499] ? watchdog+0x40/0x40 [ 5528.210916] ? hrtimer_interrupt+0x128/0x2c0 [ 5528.213016] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5528.215163] ? apic_timer_interrupt+0xf/0x20 [ 5528.217278] [ 5528.218172] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5528.220853] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5528.222801] do_raw_spin_lock+0x177/0x190 [ 5528.225296] _raw_spin_lock+0x21/0x30 [ 5528.227074] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5528.229827] process_one_work+0x2c8/0x700 [ 5528.231648] worker_thread+0x296/0x6e0 [ 5528.233852] ? rescuer_thread+0x570/0x570 [ 5528.238378] kthread+0x1d1/0x200 [ 5528.240144] ? set_kthread_struct+0x70/0x70 [ 5528.242238] ret_from_fork+0x1f/0x30 [ 5552.057556] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [writemany:36102] [ 5552.060068] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ptlrpcd_rcv:2372] [ 5552.062030] Modules linked in: [ 5552.070942] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5552.071020] CPU: 1 PID: 2372 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5552.071028] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5552.071031] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5552.071047] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a a7 66 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5552.071052] RSP: 0018:ffffbad3419cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5552.071057] RAX: 0000000000000003 RBX: ffff99374626eae0 RCX: ffff99378209e480 [ 5552.071059] RDX: 0000000000007cf2 RSI: 0000000000000003 RDI: ffff99374626eae0 [ 5552.071062] RBP: ffff9937820b4140 R08: 0000000000000008 R09: 00000000000000ac [ 5552.071064] R10: ffffffff9b008b80 R11: ffffbad3419cf8f0 R12: 0000000000080000 [ 5552.071066] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5552.071069] FS: 0000000000000000(0000) GS:ffff993782080000(0000) knlGS:0000000000000000 [ 5552.071072] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5552.071075] CR2: 000055b914ab0890 CR3: 0000000022616006 CR4: 0000000000170ee0 [ 5552.071083] Call Trace: [ 5552.071089] [ 5552.071094] ? show_regs.cold.9+0x22/0x2f [ 5552.071103] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5552.071108] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5552.071113] ? watchdog+0x40/0x40 [ 5552.071117] ? hrtimer_interrupt+0x128/0x2c0 [ 5552.071121] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5552.071128] ? apic_timer_interrupt+0xf/0x20 [ 5552.071132] [ 5552.071135] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5552.071139] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5552.071144] do_raw_spin_lock+0x177/0x190 [ 5552.071148] _raw_spin_lock+0x21/0x30 [ 5552.071153] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5552.071437] obd_mod_free+0x46/0x140 [ptlrpc] [ 5552.071646] mdc_commit_open+0x91/0xb0 [mdc] [ 5552.071684] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5552.071893] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5552.072094] after_reply+0xe2c/0x1780 [ptlrpc] [ 5552.072287] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5552.105900] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5552.105985] CPU: 0 PID: 36102 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5552.105991] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5552.105994] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5552.106006] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5552.106010] RSP: 0018:ffffbad340e33ba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5552.106014] RAX: 0000000000003fe3 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5552.106017] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff993782034154 [ 5552.106019] RBP: ffff993782034140 R08: 0000000000000000 R09: 0000000000000000 [ 5552.106022] R10: 000000002651961c R11: ffff993748568000 R12: 0000000000040000 [ 5552.106025] R13: ffff9937820b4140 R14: 000000000001e480 R15: ffff99378201e480 [ 5552.106028] FS: 00007fe02c8df740(0000) GS:ffff993782000000(0000) knlGS:0000000000000000 [ 5552.106032] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5552.106035] CR2: 00007fd7431b8040 CR3: 00000001012ab001 CR4: 0000000000170ef0 [ 5552.106042] Call Trace: [ 5552.106047] [ 5552.106053] ? show_regs.cold.9+0x22/0x2f [ 5552.106061] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5552.106066] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5552.118547] ? __next_timer_interrupt+0x160/0x160 [ 5552.121391] ? watchdog+0x40/0x40 [ 5552.121408] ? hrtimer_interrupt+0x128/0x2c0 [ 5552.121414] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5552.121427] ? apic_timer_interrupt+0xf/0x20 [ 5552.121431] [ 5552.121436] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5552.121442] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5552.121447] do_raw_spin_lock+0x177/0x190 [ 5552.121453] _raw_spin_lock+0x21/0x30 [ 5552.121460] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5552.121854] ? do_raw_spin_unlock+0x75/0x190 [ 5552.121861] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5552.122110] ? do_wait_intr+0xf0/0xf0 [ 5552.122118] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5552.122313] mdc_reint+0x2d/0x110 [mdc] [ 5552.122349] mdc_unlink+0x658/0x870 [mdc] [ 5552.122385] lmv_unlink+0x34d/0xbd0 [lmv] [ 5552.122415] ll_unlink+0x24d/0x7f0 [lustre] [ 5552.122608] vfs_unlink+0x202/0x330 [ 5552.122621] do_unlinkat+0x2fa/0x460 [ 5552.122625] __x64_sys_unlink+0x37/0x50 [ 5552.122628] do_syscall_64+0xc1/0x440 [ 5552.122634] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5552.122639] RIP: 0033:0x7fe02c11910b [ 5552.122646] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5552.122650] RSP: 002b:00007fff45ac4bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5552.122655] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe02c11910b [ 5552.122657] RDX: 0000000000000000 RSI: 00007fff45ac4be4 RDI: 00007fff45ac4c60 [ 5552.122659] RBP: 00007fff45ac5060 R08: 00007fe02c4b1234 R09: 00007fe02c4b1260 [ 5552.122662] R10: 00007fff45ac49a2 R11: 0000000000000246 R12: 0000000000000003 [ 5552.122664] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5552.531463] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5552.535675] ? do_wait_intr+0xf0/0xf0 [ 5552.543103] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5552.548624] kthread+0x1d1/0x200 [ 5552.555481] ? set_kthread_struct+0x70/0x70 [ 5552.564647] ret_from_fork+0x1f/0x30 [ 5556.075895] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u8:3:11403] [ 5556.088482] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5556.129189] CPU: 3 PID: 11403 Comm: kworker/u8:3 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5556.137075] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5556.154799] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5556.160951] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5556.166209] Code: 4c 03 2c c5 80 58 e8 9a 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5556.186020] RSP: 0018:ffffbad341c33dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5556.192962] RAX: 0000000000007914 RBX: ffff99374626eae0 RCX: 0000000000000001 [ 5556.199367] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff9937821b4154 [ 5556.202936] RBP: ffff9937821b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5556.208236] R10: 0000000026519a80 R11: ffff993748294000 R12: 0000000000100000 [ 5556.214627] R13: ffff993782034140 R14: 000000000001e480 R15: ffff99378219e480 [ 5556.218070] FS: 0000000000000000(0000) GS:ffff993782180000(0000) knlGS:0000000000000000 [ 5556.223775] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5556.225761] CR2: 00007f5ecf4c7008 CR3: 0000000022616001 CR4: 0000000000170ee0 [ 5556.231677] Call Trace: [ 5556.233665] [ 5556.234468] ? show_regs.cold.9+0x22/0x2f [ 5556.238469] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5556.243376] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5556.246322] ? watchdog+0x40/0x40 [ 5556.248083] ? hrtimer_interrupt+0x128/0x2c0 [ 5556.253353] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5556.257152] ? apic_timer_interrupt+0xf/0x20 [ 5556.260831] [ 5556.262374] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5556.266969] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5556.270745] do_raw_spin_lock+0x177/0x190 [ 5556.274516] _raw_spin_lock+0x21/0x30 [ 5556.280289] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5556.282520] process_one_work+0x2c8/0x700 [ 5556.284467] worker_thread+0x296/0x6e0 [ 5556.286038] ? rescuer_thread+0x570/0x570 [ 5556.288184] kthread+0x1d1/0x200 [ 5556.291282] ? set_kthread_struct+0x70/0x70 [ 5556.295280] ret_from_fork+0x1f/0x30