[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 456632165 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895288K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003341] x2apic enabled [ 0.004010] Switched APIC routing to physical x2apic. [ 0.005000] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.007020] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.008012] pid_max: default: 32768 minimum: 301 [ 0.010090] LSM: Security Framework initializing [ 0.011054] Yama: becoming mindful. [ 0.012036] SELinux: Initializing. [ 0.013070] *** VALIDATE selinux *** [ 0.021791] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027322] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028160] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029099] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030118] *** VALIDATE tmpfs *** [ 0.032159] *** VALIDATE proc *** [ 0.033237] *** VALIDATE cgroup *** [ 0.034011] *** VALIDATE cgroup2 *** [ 0.035280] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036157] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038031] Spectre V2 : User space: Vulnerable [ 0.039010] Speculative Store Bypass: Vulnerable [ 0.041782] debug: unmapping init [mem 0xffffffff90059000-0xffffffff90060fff] [ 0.043146] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044541] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045017] ... version: 2 [ 0.045927] ... bit width: 48 [ 0.046011] ... generic registers: 4 [ 0.047012] ... value mask: 0000ffffffffffff [ 0.048014] ... max period: 00007fffffffffff [ 0.049013] ... fixed-purpose events: 3 [ 0.050011] ... event mask: 000000070000000f [ 0.051323] rcu: Hierarchical SRCU implementation. [ 0.053389] smp: Bringing up secondary CPUs ... [ 0.054569] x86: Booting SMP configuration: [ 0.055027] .... node #0, CPUs: #1 #2 #3 [ 0.059249] smp: Brought up 1 node, 4 CPUs [ 0.061015] smpboot: Max logical packages: 1 [ 0.062010] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.193482] node 0 deferred pages initialised in 129ms [ 0.196143] devtmpfs: initialized [ 0.198313] x86/mm: Memory block size: 128MB [ 0.201200] gcov: version magic: 0x41383552 [ 0.204357] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.208102] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.211448] pinctrl core: initialized pinctrl subsystem [ 0.213241] [ 0.213794] ************************************************************* [ 0.216017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.218014] ** ** [ 0.221017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.222016] ** ** [ 0.225065] ** This means that this kernel is built to expose internal ** [ 0.227015] ** IOMMU data structures, which may compromise security on ** [ 0.229017] ** your system. ** [ 0.231017] ** ** [ 0.234018] ** If you see this message and you are not debugging the ** [ 0.237020] ** kernel, report this immediately to your vendor! ** [ 0.240020] ** ** [ 0.241016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.243018] ************************************************************* [ 0.246960] NET: Registered protocol family 16 [ 0.249531] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.253093] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.255088] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.259157] cpuidle: using governor menu [ 0.261046] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.262926] PCI: Using configuration type 1 for base access [ 0.263000] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.272055] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.274042] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.279187] cryptd: max_cpu_qlen set to 1000 [ 0.282379] ACPI: Added _OSI(Module Device) [ 0.284016] ACPI: Added _OSI(Processor Device) [ 0.285015] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.287014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.292706] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.299637] ACPI: Interpreter enabled [ 0.301067] ACPI: PM: (supports S0 S3 S4 S5) [ 0.302000] ACPI: Using IOAPIC for interrupt routing [ 0.302000] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.305572] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.316824] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.318049] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.321026] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.324109] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.329607] acpiphp: Slot [2] registered [ 0.331188] acpiphp: Slot [5] registered [ 0.333150] acpiphp: Slot [6] registered [ 0.334215] acpiphp: Slot [3] registered [ 0.336106] acpiphp: Slot [4] registered [ 0.337169] acpiphp: Slot [7] registered [ 0.339114] acpiphp: Slot [8] registered [ 0.340106] acpiphp: Slot [9] registered [ 0.342130] acpiphp: Slot [10] registered [ 0.343139] acpiphp: Slot [11] registered [ 0.344096] acpiphp: Slot [12] registered [ 0.346099] acpiphp: Slot [13] registered [ 0.347090] acpiphp: Slot [14] registered [ 0.348144] acpiphp: Slot [15] registered [ 0.350128] acpiphp: Slot [16] registered [ 0.351126] acpiphp: Slot [17] registered [ 0.352117] acpiphp: Slot [18] registered [ 0.353117] acpiphp: Slot [19] registered [ 0.355098] acpiphp: Slot [20] registered [ 0.356097] acpiphp: Slot [21] registered [ 0.357099] acpiphp: Slot [22] registered [ 0.359283] acpiphp: Slot [23] registered [ 0.360099] acpiphp: Slot [24] registered [ 0.362146] acpiphp: Slot [25] registered [ 0.364171] acpiphp: Slot [26] registered [ 0.366146] acpiphp: Slot [27] registered [ 0.368133] acpiphp: Slot [28] registered [ 0.369093] acpiphp: Slot [29] registered [ 0.370077] acpiphp: Slot [30] registered [ 0.371075] acpiphp: Slot [31] registered [ 0.372065] PCI host bridge to bus 0000:00 [ 0.373018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.376027] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.378022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.380027] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.381022] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.385035] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.386230] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.390284] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.393576] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.402021] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.407756] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.411024] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.413015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.415019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.417551] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.420756] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.423051] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.426945] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.430916] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.441020] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.446026] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.451754] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.457015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.465022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.476019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.487131] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.495019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.505021] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.539024] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.553411] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.556421] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.558417] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.561322] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.563366] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.569042] iommu: Default domain type: Passthrough [ 0.572440] SCSI subsystem initialized [ 0.574136] ACPI: bus type USB registered [ 0.575104] usbcore: registered new interface driver usbfs [ 0.576068] usbcore: registered new interface driver hub [ 0.578071] usbcore: registered new device driver usb [ 0.579225] pps_core: LinuxPPS API ver. 1 registered [ 0.581011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.583117] PTP clock support registered [ 0.585106] EDAC MC: Ver: 3.0.0 [ 0.586600] PCI: Using ACPI for IRQ routing [ 0.589154] NetLabel: Initializing [ 0.591016] NetLabel: domain hash size = 128 [ 0.592013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.595211] NetLabel: unlabeled traffic allowed by default [ 0.599129] vgaarb: loaded [ 0.602369] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.603017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.607300] clocksource: Switched to clocksource kvm-clock [ 0.732305] VFS: Disk quotas dquot_6.6.0 [ 0.734338] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.737091] *** VALIDATE ramfs *** [ 0.738297] *** VALIDATE hugetlbfs *** [ 0.740599] pnp: PnP ACPI init [ 0.743250] pnp: PnP ACPI: found 6 devices [ 0.762752] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.766678] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.768623] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.770664] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.772979] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.775245] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.778068] NET: Registered protocol family 2 [ 0.780360] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.785859] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.789859] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.795405] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.798937] TCP: Hash tables configured (established 65536 bind 65536) [ 0.802141] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.805679] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.808770] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.812584] NET: Registered protocol family 1 [ 0.816107] RPC: Registered named UNIX socket transport module. [ 0.819360] RPC: Registered udp transport module. [ 0.820865] RPC: Registered tcp transport module. [ 0.822617] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.824672] NET: Registered protocol family 44 [ 0.826344] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.828210] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.830100] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.832315] PCI: CLS 0 bytes, default 64 [ 0.833745] Unpacking initramfs... [ 2.269395] debug: unmapping init [mem 0xffff898e3cc64000-0xffff898e3ffcffff] [ 2.272667] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.275246] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.279701] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 2.831621] Initialise system trusted keyrings [ 2.834621] Key type blacklist registered [ 2.839245] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.849344] zbud: loaded [ 2.853675] *** VALIDATE nfs *** [ 2.855312] *** VALIDATE nfs4 *** [ 2.860330] pstore: using deflate compression [ 2.863835] Platform Keyring initialized [ 2.999413] NET: Registered protocol family 38 [ 3.001481] Key type asymmetric registered [ 3.003024] Asymmetric key parser 'x509' registered [ 3.004679] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.007114] io scheduler mq-deadline registered [ 3.008614] io scheduler kyber registered [ 3.010266] io scheduler bfq registered [ 3.012491] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.015622] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.018389] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.021372] ACPI: Power Button [PWRF] [ 3.030762] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.038804] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.050177] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.076546] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.104783] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.111526] Non-volatile memory driver v1.3 [ 3.113840] Linux agpgart interface v0.103 [ 3.145302] virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.149473] vda: detected capacity change from 0 to 74702848 [ 3.163406] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.166154] vdb: detected capacity change from 0 to 1073741824 [ 3.173575] libphy: Fixed MDIO Bus: probed [ 3.182530] usbcore: registered new interface driver usbserial_generic [ 3.185075] usbserial: USB Serial support registered for generic [ 3.186937] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.192098] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.193416] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.195770] mousedev: PS/2 mouse device common for all mice [ 3.198309] rtc_cmos 00:05: RTC can wake from S4 [ 3.200972] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.203288] rtc_cmos 00:05: registered as rtc0 [ 3.206394] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.209408] intel_pstate: CPU model not supported [ 3.212926] hid: raw HID events driver (C) Jiri Kosina [ 3.215525] usbcore: registered new interface driver usbhid [ 3.217447] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.217511] usbhid: USB HID core driver [ 3.221746] drop_monitor: Initializing network drop monitor service [ 3.224374] Initializing XFRM netlink socket [ 3.224439] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.226709] NET: Registered protocol family 10 [ 3.232235] Segment Routing with IPv6 [ 3.233542] NET: Registered protocol family 17 [ 3.235799] mpls_gso: MPLS GSO support [ 3.242680] RAS: Correctable Errors collector initialized. [ 3.244916] AVX version of gcm_enc/dec engaged. [ 3.246900] AES CTR mode by8 optimization enabled [ 3.326645] sched_clock: Marking stable (3326620160, 0)->(4225084870, -898464710) [ 3.330245] registered taskstats version 1 [ 3.332190] Loading compiled-in X.509 certificates [ 3.333778] zswap: loaded using pool lzo/zbud [ 3.358752] Key type big_key registered [ 3.372564] Key type encrypted registered [ 3.374587] ima: No TPM chip found, activating TPM-bypass! [ 3.376240] ima: Allocated hash algorithm: sha1 [ 3.377896] ima: No architecture policies found [ 3.379807] evm: Initialising EVM extended attributes: [ 3.381774] evm: security.selinux [ 3.382922] evm: security.ima [ 3.384061] evm: security.capability [ 3.385576] evm: HMAC attrs: 0x1 [ 3.388238] rtc_cmos 00:05: setting system clock to 2026-08-15 13:20:35 UTC (1786800035) [ 3.394166] debug: unmapping init [mem 0xffffffff91003000-0xffffffff911fffff] [ 3.397409] debug: unmapping init [mem 0xffffffff8fd82000-0xffffffff90058fff] [ 3.406686] Write protecting the kernel read-only data: 28672k [ 3.409847] debug: unmapping init [mem 0xffffffff8e403000-0xffffffff8e5fffff] [ 3.412341] debug: unmapping init [mem 0xffffffff8ed14000-0xffffffff8edfffff] [ 3.457761] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.466261] systemd[1]: Detected virtualization kvm. [ 3.468459] systemd[1]: Detected architecture x86-64. [ 3.470332] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.500791] systemd[1]: No hostname configured. [ 3.502774] systemd[1]: Set hostname to . [ 3.504701] random: systemd: uninitialized urandom read (16 bytes read) [ 3.506859] systemd[1]: Initializing machine ID from random generator. [ 3.556311] random: ln: uninitialized urandom read (6 bytes read) [ 3.637725] random: systemd: uninitialized urandom read (16 bytes read) [ 3.641544] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.648257] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.654000] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Paths. Starting Setup Virtual Console... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Timers. Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.224721] device-mapper: uevent: version 1.0.3 [ 4.226488] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.905907] virtio_net virtio0 ens2: renamed from eth0 [ 4.995599] scsi host0: ata_piix [ 5.032372] scsi host1: ata_piix [ 5.034126] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 5.036706] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 6.085068] random: fast init done [ 8.671606] dracut-initqueue[578]: RTNETLINK answers: File exists [ 9.896293] random: crng init done [ 9.897849] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.249187] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.435171] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.700293] SELinux: Disabled at runtime. [ 11.763227] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.771615] systemd[1]: Detected virtualization kvm. [ 11.773474] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.312460] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.316151] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.323889] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.328340] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.333787] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.346672] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.354744] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on Process Core Dump Socket. Activating swap /dev/disk/by-label/SWAP... Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. [ OK ] Reached target rpc_pipefs.target. Mounting Huge Pages File System... [ 12.424488] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting POSIX Message Queue File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Paths. Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Mounting Kernel Debug File System... [ OK ] Reached target RPC Port Mapper. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Initrd File Systems. [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.888339] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.200792] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.244865] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.317644] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.332496] EDAC sbridge: Ver: 1.1.2 [ 14.419355] Key type dns_resolver registered [ 14.717272] NFS: Registering the id_resolver key type [ 14.719717] Key type id_resolver registered [ 14.721558] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. [ OK ] Started Authorization Manager. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg434-client login: [ 54.194798] libcfs: loading out-of-tree module taints kernel. [ 54.253652] Key type ._llcrypt registered [ 54.256308] Key type .llcrypt registered [ 54.925151] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 54.938432] alg: No test for adler32 (adler32-zlib) [ 56.490987] Lustre: Lustre: Build Version: 2.17.57_1_g05abe65 [ 57.759078] LNet: Added LNI 192.168.204.34@tcp [8/256/0/180] [ 59.647169] Key type lgssc registered [ 61.772613] Lustre: Echo OBD driver; http://www.lustre.org/ [ 90.470015] hrtimer: interrupt took 6017950 ns [ 221.643144] Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 [ 226.790666] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 243.142729] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing check_logdir /tmp/testlogs/ [ 247.263205] Lustre: lustre-OST0000-osc-ffff898e8e663800: disconnect after 23s idle [ 248.153865] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing yml_node [ 251.572833] Lustre: DEBUG MARKER: Client: 2.17.57.1 [ 253.722787] Lustre: DEBUG MARKER: MDS: 2.17.57.1 [ 255.588697] Lustre: DEBUG MARKER: OSS: 2.17.57.1 [ 257.059536] Lustre: DEBUG MARKER: -----============= acceptance-small: recovery-small ============----- Sat Aug 15 09:24:47 EDT 2026 [ 272.672761] Lustre: DEBUG MARKER: excepting tests: 136 [ 274.900107] Lustre: DEBUG MARKER: === recovery-small: start setup 09:25:05 (1786800305) === [ 281.038058] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing check_config_client /mnt/lustre [ 302.396572] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 318.018092] Lustre: DEBUG MARKER: === recovery-small: finish setup 09:25:48 (1786800348) === [ 319.406886] Lustre: DEBUG MARKER: == recovery-small test 1: create, chmod, stat: drop req, drop rep ========================================================== 09:25:50 (1786800350) [ 337.375298] Lustre: 10841:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786800353/real 1786800353] req@ffff898ea8d2ce00 x1873595690138624/t0(0) o700->lustre-MDT0000-mdc-ffff898e8e663800@192.168.204.134@tcp:30/10 lens 264/248 e 0 to 1 dl 1786800369 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'mcreate.0' uid:0 gid:0 projid:4294967295 [ 337.407449] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 337.448578] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 356.322355] Lustre: 10862:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786800372/real 1786800372] req@ffff898e83108e00 x1873595690140928/t0(0) o36->lustre-MDT0000-mdc-ffff898e8e663800@192.168.204.134@tcp:12/10 lens 520/576 e 0 to 1 dl 1786800388 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'mcreate.0' uid:0 gid:0 projid:4294967295 [ 356.393895] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 356.452438] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 376.287273] Lustre: 10905:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786800392/real 1786800392] req@ffff898e83108e00 x1873595690143872/t0(0) o36->lustre-MDT0000-mdc-ffff898e8e663800@192.168.204.134@tcp:12/10 lens 488/512 e 0 to 1 dl 1786800408 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'tchmod.0' uid:0 gid:0 projid:4294967295 [ 376.312486] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 376.352928] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 395.231218] Lustre: 10931:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786800411/real 1786800411] req@ffff898ea8d2dc00 x1873595690145664/t0(0) o34->lustre-MDT0000-mdc-ffff898e8e663800@192.168.204.134@tcp:12/10 lens 472/728 e 0 to 1 dl 1786800427 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'statone.0' uid:0 gid:0 projid:0 [ 395.261885] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 395.306341] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 413.663165] Lustre: 10952:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786800429/real 1786800429] req@ffff898e84da8000 x1873595690147456/t0(0) o34->lustre-MDT0000-mdc-ffff898e8e663800@192.168.204.134@tcp:12/10 lens 472/728 e 0 to 1 dl 1786800445 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'statone.0' uid:0 gid:0 projid:0 [ 413.687338] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 413.721531] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 423.608264] Lustre: DEBUG MARKER: == recovery-small test 4: open: drop req, drop rep ======= 09:27:33 (1786800453) [ 441.312970] Lustre: 11564:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786800457/real 1786800457] req@ffff898e880eca80 x1873595690150656/t0(0) o101->lustre-MDT0000-mdc-ffff898e8e663800@192.168.204.134@tcp:12/10 lens 576/1152 e 0 to 1 dl 1786800473 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 [ 441.334427] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 441.382649] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 450.722710] Lustre: DEBUG MARKER: == recovery-small test 5: rename: drop req, drop rep ===== 09:28:01 (1786800481) [ 468.447211] Lustre: 12189:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786800484/real 1786800484] req@ffff898ea8d2dc00 x1873595690156160/t0(0) o101->lustre-MDT0000-mdc-ffff898e8e663800@192.168.204.134@tcp:12/10 lens 576/1152 e 0 to 1 dl 1786800500 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'mv.0' uid:0 gid:0 projid:0 [ 468.479108] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 468.526891] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 494.790639] Lustre: DEBUG MARKER: == recovery-small test 6: link, unlink: drop req, drop rep ========================================================== 09:28:45 (1786800525) [ 512.479434] Lustre: 12832:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786800528/real 1786800528] req@ffff898e83109180 x1873595690164992/t0(0) o101->lustre-MDT0000-mdc-ffff898e8e663800@192.168.204.134@tcp:12/10 lens 576/1152 e 0 to 1 dl 1786800544 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'link.0' uid:0 gid:0 projid:0 [ 512.502600] Lustre: 12832:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 512.510055] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 512.526296] Lustre: Skipped 1 previous similar message [ 512.571304] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 512.582461] Lustre: Skipped 1 previous similar message [ 517.599666] Lustre: lustre-OST0000-osc-ffff898e8e663800: disconnect after 23s idle [ 517.616424] Lustre: Skipped 1 previous similar message [ 578.209112] Lustre: DEBUG MARKER: == recovery-small test 8: touch: drop rep (bug 1423) ===== 09:30:08 (1786800608) [ 595.423217] Lustre: 13504:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786800611/real 1786800611] req@ffff898ea8d2f480 x1873595690178304/t0(0) o36->lustre-MDT0000-mdc-ffff898e8e663800@192.168.204.134@tcp:12/10 lens 488/512 e 0 to 1 dl 1786800627 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:4294967295 [ 595.480487] Lustre: 13504:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [ 595.484472] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 595.490209] Lustre: Skipped 3 previous similar messages [ 595.510093] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 595.515566] Lustre: Skipped 3 previous similar messages [ 603.608903] Lustre: DEBUG MARKER: == recovery-small test 9: pause bulk on OST (bug 1420) === 09:30:34 (1786800634) [ 619.547663] Lustre: DEBUG MARKER: == recovery-small test 10a: finish request on server after client eviction (bug 1521) ========================================================== 09:30:50 (1786800650) [ 620.158726] Lustre: *** cfs_fail_loc=305, val=0*** [ 635.424199] Lustre: *** cfs_fail_loc=305, val=0*** [ 641.503671] Lustre: lustre-OST0000-osc-ffff898e8e663800: disconnect after 24s idle [ 651.826932] Lustre: *** cfs_fail_loc=305, val=0*** [ 668.184823] Lustre: *** cfs_fail_loc=305, val=0*** [ 683.533210] Lustre: *** cfs_fail_loc=305, val=0*** [ 699.933487] Lustre: *** cfs_fail_loc=305, val=0*** [ 716.286655] Lustre: *** cfs_fail_loc=305, val=0*** [ 733.067484] LustreError: lustre-MDT0000-mdc-ffff898e8e663800: operation ldlm_enqueue to node 192.168.204.134@tcp failed: rc = -107 [ 733.080595] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 733.100743] LustreError: lustre-MDT0000-mdc-ffff898e8e663800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. [ 733.128583] Lustre: lustre-MDT0000-mdc-ffff898e8e663800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 743.891595] Lustre: DEBUG MARKER: == recovery-small test 10b: re-send BL AST =============== 09:32:54 (1786800774) [ 759.263602] Lustre: lustre-OST0000-osc-ffff898e8e663800: disconnect after 20s idle [ 768.976498] Lustre: DEBUG MARKER: == recovery-small test 10c: re-send BL AST vs reconnect race (LU-5569) ========================================================== 09:33:19 (1786800799) [ 769.694604] Lustre: *** cfs_fail_loc=305, val=0*** [ 769.697622] Lustre: Skipped 1 previous similar message [ 778.168843] Lustre: DEBUG MARKER: == recovery-small test 10d: test failed blocking ast ===== 09:33:28 (1786800808) [ 782.247080] Lustre: Unmounted lustre-client [ 782.903725] Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 [ 784.855485] LustreError: lustre-OST0000-osc-ffff898e829ef800: operation ost_statfs to node 192.168.204.134@tcp failed: rc = -107 [ 784.868165] LustreError: lustre-OST0000-osc-ffff898e829ef800: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 784.884894] Lustre: 2390:0:(llite_lib.c:4340:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.204.134@tcp:/lustre/fid: [0x200000404:0x1:0x0]/ may get corrupted (rc -108) [ 786.821752] Lustre: Unmounted lustre-client [ 793.494889] Lustre: DEBUG MARKER: == recovery-small test 10e: re-send BL AST vs reconnect race 2 ========================================================== 09:33:44 (1786800824) [ 795.371800] Lustre: DEBUG MARKER: SKIP: recovery-small test_10e need two clients [ 797.751617] Lustre: DEBUG MARKER: == recovery-small test 11: wake up a thread waiting for completion after eviction (b=2460) ========================================================== 09:33:47 (1786800827) [ 823.637286] Lustre: DEBUG MARKER: == recovery-small test 12: recover from timed out resend in ptlrpcd (b=2494) ========================================================== 09:34:14 (1786800854) [ 823.863062] Lustre: DEBUG MARKER: multiop /mnt/lustre/f12.recovery-small OS_c [ 841.695279] Lustre: 18561:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786800857/real 1786800857] req@ffff898e84da9880 x1873595690250880/t0(0) o35->lustre-MDT0000-mdc-ffff898e829ef800@192.168.204.134@tcp:23/10 lens 392/624 e 0 to 1 dl 1786800873 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 [ 868.680912] Lustre: DEBUG MARKER: == recovery-small test 13: mdc_readpage restart test (bug 1138) ========================================================== 09:34:59 (1786800899) [ 894.220828] Lustre: DEBUG MARKER: == recovery-small test 14: mdc_readpage resend test (bug 1138) ========================================================== 09:35:25 (1786800925) [ 903.050426] Lustre: DEBUG MARKER: == recovery-small test 15: failed open (-ENOMEM) ========= 09:35:33 (1786800933) [ 911.396285] Lustre: DEBUG MARKER: == recovery-small test 16: timeout bulk put, don't evict client (2732) ========================================================== 09:35:42 (1786800942) [ 960.165646] Lustre: DEBUG MARKER: == recovery-small test 17a: timeout bulk get, don't evict client (2732) ========================================================== 09:36:30 (1786800990) [ 1014.735837] Lustre: DEBUG MARKER: == recovery-small test 17b: timeout bulk get, dont evict client (3582) ========================================================== 09:37:25 (1786801045) [ 1016.211028] Lustre: DEBUG MARKER: SKIP: recovery-small test_17b Needs multiple clients [ 1018.122733] Lustre: DEBUG MARKER: == recovery-small test 18a: manual ost invalidate clears page cache immediately ========================================================== 09:37:28 (1786801048) [ 1018.954724] Lustre: setting import lustre-OST0001_UUID INACTIVE by administrator request [ 1019.004741] Lustre: lustre-OST0001-osc-ffff898e829ef800: Connection to lustre-OST0001 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 1019.018320] Lustre: Skipped 5 previous similar messages [ 1019.027475] LustreError: lustre-OST0001-osc-ffff898e829ef800: This client was evicted by lustre-OST0001; in progress operations using this service will fail. [ 1026.166110] Lustre: DEBUG MARKER: == recovery-small test 18b: eviction and reconnect clears page cache (2766) ========================================================== 09:37:36 (1786801056) [ 1032.695481] LustreError: lustre-OST0000-osc-ffff898e829ef800: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1032.706739] Lustre: lustre-OST0000-osc-ffff898e829ef800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 1032.711566] Lustre: Skipped 4 previous similar messages [ 1056.565717] Lustre: DEBUG MARKER: == recovery-small test 18c: Dropped connect reply after eviction handing (14755) ========================================================== 09:38:07 (1786801087) [ 1058.297342] LustreError: lustre-OST0000-osc-ffff898e829ef800: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1080.149694] Lustre: DEBUG MARKER: == recovery-small test 19a: test expired_lock_main on mds (2867) ========================================================== 09:38:30 (1786801110) [ 1080.873114] Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 [ 1080.881220] Lustre: Skipped 1 previous similar message [ 1099.231535] Lustre: 2390:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786801115/real 1786801115] req@ffff898ea8d2e680 x1873595690317312/t0(0) o103->lustre-MDT0000-mdc-ffff898e829ef800@192.168.204.134@tcp:17/18 lens 328/224 e 0 to 1 dl 1786801131 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 [ 1099.269850] Lustre: 2390:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 2 previous similar messages [ 1187.463663] LustreError: lustre-MDT0000-mdc-ffff898e829ef800: operation ldlm_enqueue to node 192.168.204.134@tcp failed: rc = -107 [ 1187.503626] LustreError: lustre-MDT0000-mdc-ffff898e829ef800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. [ 1187.524721] LustreError: 24895:0:(file.c:6156:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -5 [ 1189.684856] Lustre: Unmounted lustre-client [ 1197.625223] Lustre: DEBUG MARKER: == recovery-small test 19b: test expired_lock_main on ost (2867) ========================================================== 09:40:28 (1786801228) [ 1198.304445] Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 [ 1307.150116] LustreError: lustre-OST0001-osc-ffff898e829ef800: This client was evicted by lustre-OST0001; in progress operations using this service will fail. [ 1308.754108] Lustre: Unmounted lustre-client [ 1317.143454] Lustre: DEBUG MARKER: == recovery-small test 19c: check reconnect and lock resend do not trigger expired_lock_main ========================================================== 09:42:27 (1786801347) [ 1317.869753] Lustre: Mounted lustre-client - version 2.17.57_1_g05abe65 [ 1324.261852] Lustre: Unmounted lustre-client [ 1337.144316] Lustre: DEBUG MARKER: == recovery-small test 20a: ldlm_handle_enqueue error (should return error) ========================================================== 09:42:47 (1786801367) [ 1339.248946] LustreError: lustre-OST0000-osc-ffff898e829ef800: operation ldlm_enqueue to node 192.168.204.134@tcp failed: rc = -12 [ 1347.037085] Lustre: DEBUG MARKER: == recovery-small test 20b: ldlm_handle_enqueue error (should return error) ========================================================== 09:42:57 (1786801377) [ 1348.801725] LustreError: lustre-OST0000-osc-ffff898e829ef800: operation ldlm_enqueue to node 192.168.204.134@tcp failed: rc = -12 [ 1356.183504] Lustre: DEBUG MARKER: == recovery-small test 21a: drop close request while close and open are both in flight ========================================================== 09:43:06 (1786801386) [ 1387.953354] Lustre: DEBUG MARKER: == recovery-small test 21b: drop open request while close and open are both in flight ========================================================== 09:43:38 (1786801418) [ 1535.316464] Lustre: DEBUG MARKER: == recovery-small test 21c: drop both request while close and open are both in flight ========================================================== 09:46:05 (1786801565) [ 1558.497855] Lustre: lustre-MDT0000-mdc-ffff898e829ef800: Connection to lustre-MDT0000 (at 192.168.204.134@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 1558.525246] Lustre: Skipped 18 previous similar messages [ 1558.561437] Lustre: lustre-MDT0000-mdc-ffff898e829ef800: Connection restored to 192.168.204.134@tcp (at 192.168.204.134@tcp) [ 1558.574563] Lustre: Skipped 16 previous similar messages [ 1568.168645] Lustre: DEBUG MARKER: == recovery-small test 21d: drop close reply while close and open are both in flight ========================================================== 09:46:38 (1786801598) [ 1599.963683] Lustre: DEBUG MARKER: == recovery-small test 21e: drop open reply while close and open are both in flight ========================================================== 09:47:10 (1786801630) [ 1738.719166] Lustre: 30731:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786801633/real 1786801633] req@ffff898ea8d2f480 x1873595690452864/t0(0) o36->lustre-MDT0000-mdc-ffff898e829ef800@192.168.204.134@tcp:12/10 lens 488/512 e 0 to 1 dl 1786801770 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:4294967295 [ 1738.763477] Lustre: 30731:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 15 previous similar messages [ 1740.453154] Lustre: DEBUG MARKER: == recovery-small test 21f: drop both reply while close and open are both in flight ========================================================== 09:49:31 (1786801771) [ 1768.164327] Lustre: DEBUG MARKER: == recovery-small test 21g: drop open reply and close request while close and open are both in flight ========================================================== 09:49:59 (1786801799) [ 1794.989250] Lustre: DEBUG MARKER: == recovery-small test 21h: drop open request and close reply while close and open are both in flight ========================================================== 09:50:25 (1786801825) [ 1821.102408] Lustre: DEBUG MARKER: == recovery-small test 22: drop close request and do mknod ========================================================== 09:50:51 (1786801851) [ 1846.123138] Lustre: DEBUG MARKER: == recovery-small test 23: client hang when close a file after mds crash ========================================================== 09:51:16 (1786801876) [ 1875.423715] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 192.168.204.134@tcp) was lost; in progress operations using this service will fail [ 1875.459292] Lustre: Evicted from MGS (at 192.168.204.134@tcp) after server handle changed from 0xf004fe7c74fa1190 to 0xf004fe7c74fa3809 [ 1877.307276] Lustre: 16817:0:(mgc_request.c:1899:mgc_process_log()) MGC192.168.204.134@tcp: IR log lustre-cliir failed, not fatal: rc = -2 [ 1898.787802] Lustre: DEBUG MARKER: oleg434-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1900.239896] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1909.168469] Lustre: DEBUG MARKER: == recovery-small test 24a: fsync error (should return error) ========================================================== 09:52:19 (1786801939) [ 1911.508583] LustreError: lustre-OST0000-osc-ffff898e829ef800: operation ost_write to node 192.168.204.134@tcp failed: rc = -107 [ 1911.551375] LustreError: lustre-OST0000-osc-ffff898e829ef800: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1911.571283] Lustre: 2388:0:(llite_lib.c:4340:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.204.134@tcp:/lustre/fid: [0x240000402:0x8:0x0]// may get corrupted (rc -5) [ 1921.528639] Lustre: DEBUG MARKER: == recovery-small test 24b: test dirty page discard due to client eviction ========================================================== 09:52:31 (1786801951) [ 1923.455813] LustreError: lustre-OST0000-osc-ffff898e829ef800: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1923.470378] Lustre: 2390:0:(llite_lib.c:4340:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.204.134@tcp:/lustre/fid: [0x200000405:0x2e:0x0]// may get corrupted (rc -5) [ 1923.493400] Lustre: 2391:0:(llite_lib.c:4340:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.204.134@tcp:/lustre/fid: [0x200000405:0x2f:0x0]// may get corrupted (rc -108) [ 1930.298983] Lustre: DEBUG MARKER: == recovery-small test 26a: evict dead exports =========== 09:52:40 (1786801960) [ 1932.912279] Lustre: DEBUG MARKER: SKIP: recovery-small test_26a msg and ost1 are at the same node [ 1934.958418] Lustre: DEBUG MARKER: == recovery-small test 26b: evict dead exports =========== 09:52:45 (1786801965) [ 1937.298865] Lustre: DEBUG MARKER: SKIP: recovery-small test_26b msg and ost1 are at the same node [ 1939.363557] Lustre: DEBUG MARKER: == recovery-small test 27: fail LOV while using OSC's ==== 09:52:49 (1786801969) [ 1942.883153] LustreError: lustre-MDT0000-mdc-ffff898e829ef800: operation mds_reint to node 192.168.204.134@tcp failed: rc = -19 [ 1942.893907] LustreError: Skipped 1 previous similar message [ 1963.487710] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 192.168.204.134@tcp) was lost; in progress operations using this service will fail [ 1963.530544] Lustre: Evicted from MGS (at 192.168.204.134@tcp) after server handle changed from 0xf004fe7c74fa3809 to 0xf004fe7c74fa523b [ 1965.057189] Lustre: 16817:0:(mgc_request.c:1899:mgc_process_log()) MGC192.168.204.134@tcp: IR log lustre-cliir failed, not fatal: rc = -2 [ 2075.484636] LustreError: lustre-MDT0000-mdc-ffff898e829ef800: operation mds_reint to node 192.168.204.134@tcp failed: rc = -19 [ 2092.511827] LustreError: MGC192.168.204.134@tcp: Connection to MGS (at 192.168.204.134@tcp) was lost; in progress operations using this service will fail [ 2097.669146] BUG: spinlock recursion on CPU#0, ptlrpcd_rcv/2387 [ 2097.673523] lock: 0xffff898e84d1d2e0, .magic: dead4ead, .owner: ptlrpcd_rcv/2387, .owner_cpu: 0 [ 2097.681827] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 [ 2097.703343] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2097.716495] Call Trace: [ 2097.723922] ? dump_stack+0xbb/0x10e [ 2097.734687] ? spin_dump.cold.3+0x32/0x65 [ 2097.742324] ? do_raw_spin_lock+0x10f/0x190 [ 2097.751255] ? _raw_spin_lock+0x21/0x30 [ 2097.758665] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2097.773322] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2097.781708] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2097.784942] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2097.793894] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2097.807027] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2097.818712] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2097.830376] ? __next_timer_interrupt+0x160/0x160 [ 2097.839643] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2097.847608] ? do_wait_intr+0xf0/0xf0 [ 2097.853714] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2097.857940] ? kthread+0x1d1/0x200 [ 2097.863515] ? set_kthread_struct+0x70/0x70 [ 2097.866526] ? ret_from_fork+0x1f/0x30 [ 2124.054515] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2124.059055] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2124.066225] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) [ 2124.074065] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2124.074186] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 [ 2124.074193] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2124.074196] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2124.074219] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2124.074223] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2124.074472] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2124.074475] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2124.074477] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2124.074479] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2124.074481] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2124.074484] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2124.074487] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2124.074490] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2124.074497] Call Trace: [ 2124.077099] [ 2124.079187] obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2124.144848] ? show_regs.cold.9+0x22/0x2f [ 2124.144915] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2124.144925] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2124.144931] ? watchdog+0x40/0x40 [ 2124.144935] ? hrtimer_interrupt+0x128/0x2c0 [ 2124.144939] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2124.144948] ? apic_timer_interrupt+0xf/0x20 [ 2124.144961] [ 2124.144967] ? native_safe_halt+0x2e/0x50 [ 2124.144972] kvm_wait+0xcd/0xe0 [ 2124.144981] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2124.144988] do_raw_spin_lock+0x177/0x190 [ 2124.144993] _raw_spin_lock+0x21/0x30 [ 2124.144999] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2124.145548] ? do_raw_spin_unlock+0x75/0x190 [ 2124.145554] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2124.146108] ? do_wait_intr+0xf0/0xf0 [ 2124.146116] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2124.147664] mdc_reint+0x2d/0x110 [mdc] [ 2124.147719] mdc_unlink+0x658/0x870 [mdc] [ 2124.147750] lmv_unlink+0x34d/0xbd0 [lmv] [ 2124.147789] ll_unlink+0x24d/0x7f0 [lustre] [ 2124.147986] vfs_unlink+0x202/0x330 [ 2124.147995] do_unlinkat+0x2fa/0x460 [ 2124.148000] __x64_sys_unlink+0x37/0x50 [ 2124.148004] do_syscall_64+0xc1/0x440 [ 2124.148010] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2124.148020] RIP: 0033:0x7f33957fc10b [ 2124.148028] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2124.148032] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2124.148037] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2124.148040] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2124.148043] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2124.148045] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2124.148047] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2124.435236] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2124.443943] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2124.453907] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2124.458893] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2124.473671] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2124.477545] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2124.481145] RDX: 00000000000069bc RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2124.487185] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2124.492421] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2124.496650] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2124.501225] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2124.505669] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2124.507599] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2124.510780] Call Trace: [ 2124.511681] [ 2124.515189] ? show_regs.cold.9+0x22/0x2f [ 2124.517506] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2124.521691] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2124.525478] ? watchdog+0x40/0x40 [ 2124.529437] ? hrtimer_interrupt+0x128/0x2c0 [ 2124.532169] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2124.534640] ? apic_timer_interrupt+0xf/0x20 [ 2124.537396] [ 2124.538278] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2124.542167] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2124.553215] do_raw_spin_lock+0x177/0x190 [ 2124.555856] _raw_spin_lock+0x21/0x30 [ 2124.558922] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2124.565933] obd_mod_free+0x46/0x140 [ptlrpc] [ 2124.574872] mdc_commit_open+0x91/0xb0 [mdc] [ 2124.577680] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2124.583734] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2124.590991] after_reply+0xe2c/0x1780 [ptlrpc] [ 2124.596733] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2124.602423] ? __next_timer_interrupt+0x160/0x160 [ 2124.606783] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2124.610710] ? do_wait_intr+0xf0/0xf0 [ 2124.613859] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2124.619604] kthread+0x1d1/0x200 [ 2124.623686] ? set_kthread_struct+0x70/0x70 [ 2124.630295] ret_from_fork+0x1f/0x30 [ 2128.058155] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2128.065288] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2128.119665] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2128.133031] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2128.145065] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2128.152818] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2128.158326] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2128.177307] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2128.186416] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2128.195288] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2128.203026] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2128.208643] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2128.214225] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2128.222958] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2128.232198] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2128.239738] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2128.247592] Call Trace: [ 2128.251039] [ 2128.252860] ? show_regs.cold.9+0x22/0x2f [ 2128.258134] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2128.264208] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2128.270116] ? watchdog+0x40/0x40 [ 2128.273368] ? hrtimer_interrupt+0x128/0x2c0 [ 2128.276696] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2128.283034] ? apic_timer_interrupt+0xf/0x20 [ 2128.288054] [ 2128.291388] ? native_safe_halt+0x2e/0x50 [ 2128.297420] kvm_wait+0xcd/0xe0 [ 2128.301681] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2128.307284] do_raw_spin_lock+0x177/0x190 [ 2128.315221] _raw_spin_lock+0x21/0x30 [ 2128.318941] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2128.323990] process_one_work+0x2c8/0x700 [ 2128.326949] worker_thread+0x296/0x6e0 [ 2128.333144] ? rescuer_thread+0x570/0x570 [ 2128.338881] kthread+0x1d1/0x200 [ 2128.344063] ? set_kthread_struct+0x70/0x70 [ 2128.349812] ret_from_fork+0x1f/0x30 [ 2152.053056] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2152.059030] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2152.059055] Modules linked in: [ 2152.066339] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2152.066493] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2152.066768] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2152.066771] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2152.066784] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2152.066788] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2152.066793] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2152.066795] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2152.066798] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2152.066800] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2152.066803] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2152.066806] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2152.066809] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2152.066811] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2152.066820] Call Trace: [ 2152.066826] [ 2152.066832] ? show_regs.cold.9+0x22/0x2f [ 2152.066842] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2152.066848] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2152.066852] ? watchdog+0x40/0x40 [ 2152.066856] ? hrtimer_interrupt+0x128/0x2c0 [ 2152.066860] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2152.066866] ? apic_timer_interrupt+0xf/0x20 [ 2152.066871] [ 2152.066874] ? native_safe_halt+0x2e/0x50 [ 2152.066878] kvm_wait+0xcd/0xe0 [ 2152.066886] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2152.066893] do_raw_spin_lock+0x177/0x190 [ 2152.066897] _raw_spin_lock+0x21/0x30 [ 2152.066902] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2152.068571] ? do_raw_spin_unlock+0x75/0x190 [ 2152.068583] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2152.069852] ? do_wait_intr+0xf0/0xf0 [ 2152.069862] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2152.073146] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) [ 2152.140649] mdc_reint+0x2d/0x110 [mdc] [ 2152.140707] mdc_unlink+0x658/0x870 [mdc] [ 2152.140739] lmv_unlink+0x34d/0xbd0 [lmv] [ 2152.140776] ll_unlink+0x24d/0x7f0 [lustre] [ 2152.140897] vfs_unlink+0x202/0x330 [ 2152.140906] do_unlinkat+0x2fa/0x460 [ 2152.140911] __x64_sys_unlink+0x37/0x50 [ 2152.140914] do_syscall_64+0xc1/0x440 [ 2152.140921] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2152.140930] RIP: 0033:0x7f33957fc10b [ 2152.140939] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2152.140944] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2152.140949] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2152.140952] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2152.140954] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2152.140956] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2152.140958] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2152.513415] lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2152.545924] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2152.558528] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2152.572174] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2152.576663] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2152.601039] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2152.614405] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2152.619088] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2152.630668] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2152.638556] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2152.646451] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2152.654369] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2152.665035] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2152.670711] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2152.674828] Call Trace: [ 2152.675916] [ 2152.676928] ? show_regs.cold.9+0x22/0x2f [ 2152.679618] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2152.682337] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2152.684920] ? watchdog+0x40/0x40 [ 2152.686703] ? hrtimer_interrupt+0x128/0x2c0 [ 2152.689193] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2152.692182] ? apic_timer_interrupt+0xf/0x20 [ 2152.694688] [ 2152.696200] ? native_safe_halt+0x2e/0x50 [ 2152.698079] kvm_wait+0xcd/0xe0 [ 2152.700098] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2152.702608] do_raw_spin_lock+0x177/0x190 [ 2152.704605] _raw_spin_lock+0x21/0x30 [ 2152.706236] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2152.711076] obd_mod_free+0x46/0x140 [ptlrpc] [ 2152.713825] mdc_commit_open+0x91/0xb0 [mdc] [ 2152.717052] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2152.720158] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2152.723184] after_reply+0xe2c/0x1780 [ptlrpc] [ 2152.725805] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2152.730014] ? __next_timer_interrupt+0x160/0x160 [ 2152.734150] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2152.736829] ? do_wait_intr+0xf0/0xf0 [ 2152.738722] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2152.741294] kthread+0x1d1/0x200 [ 2152.742351] ? set_kthread_struct+0x70/0x70 [ 2152.744930] ret_from_fork+0x1f/0x30 [ 2156.060869] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2156.069537] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2156.103775] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2156.110728] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2156.116236] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2156.121350] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2156.123391] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2156.133794] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2156.137231] RAX: 0000000000002a0d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2156.145755] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2156.152354] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2156.160204] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2156.163785] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2156.170909] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2156.180283] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2156.183906] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2156.190341] Call Trace: [ 2156.191272] [ 2156.192748] ? show_regs.cold.9+0x22/0x2f [ 2156.195610] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2156.199509] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2156.202557] ? watchdog+0x40/0x40 [ 2156.205974] ? hrtimer_interrupt+0x128/0x2c0 [ 2156.207484] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2156.212239] ? apic_timer_interrupt+0xf/0x20 [ 2156.215362] [ 2156.216412] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2156.221513] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2156.224256] do_raw_spin_lock+0x177/0x190 [ 2156.227536] _raw_spin_lock+0x21/0x30 [ 2156.229793] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2156.233019] process_one_work+0x2c8/0x700 [ 2156.235745] worker_thread+0x296/0x6e0 [ 2156.237936] ? rescuer_thread+0x570/0x570 [ 2156.239752] kthread+0x1d1/0x200 [ 2156.243662] ? set_kthread_struct+0x70/0x70 [ 2156.246149] ret_from_fork+0x1f/0x30 [ 2158.020735] rcu: INFO: rcu_sched self-detected stall on CPU [ 2158.033361] rcu: 0-....: (30892 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=10260 [ 2158.043175] (t=60000 jiffies g=297885 q=2241) [ 2158.045149] NMI backtrace for cpu 0 [ 2158.046519] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2158.056495] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2158.067407] Call Trace: [ 2158.071508] [ 2158.078601] ? dump_stack+0xbb/0x10e [ 2158.083670] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 2158.086640] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 2158.089619] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 2158.092140] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 2158.098595] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 2158.102009] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 2158.103572] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 2158.106657] ? update_process_times+0x6d/0xc0 [ 2158.111835] ? tick_sched_handle.isra.18+0x27/0x90 [ 2158.115076] ? tick_sched_timer+0x53/0xb0 [ 2158.117523] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2158.122820] ? tick_sched_handle.isra.18+0x90/0x90 [ 2158.127112] ? hrtimer_interrupt+0x128/0x2c0 [ 2158.130774] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2158.136412] ? apic_timer_interrupt+0xf/0x20 [ 2158.140548] [ 2158.143010] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2158.145801] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2158.149011] ? do_raw_spin_lock+0x177/0x190 [ 2158.151216] ? _raw_spin_lock+0x21/0x30 [ 2158.153388] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2158.155786] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2158.158649] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2158.161188] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2158.164555] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2158.167070] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2158.169381] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2158.172173] ? __next_timer_interrupt+0x160/0x160 [ 2158.173449] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2158.175547] ? do_wait_intr+0xf0/0xf0 [ 2158.177105] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2158.179695] ? kthread+0x1d1/0x200 [ 2158.181218] ? set_kthread_struct+0x70/0x70 [ 2158.183079] ? ret_from_fork+0x1f/0x30 [ 2180.059140] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2180.069487] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2180.119389] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2180.129329] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2180.140193] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2180.144802] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2180.163262] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2180.167934] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2180.172628] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2180.175150] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2180.183800] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2180.189240] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2180.194653] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2180.201313] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2180.209468] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2180.212798] Call Trace: [ 2180.214132] [ 2180.215608] ? show_regs.cold.9+0x22/0x2f [ 2180.218756] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2180.221091] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2180.224679] ? watchdog+0x40/0x40 [ 2180.228209] ? hrtimer_interrupt+0x128/0x2c0 [ 2180.231115] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2180.235648] ? apic_timer_interrupt+0xf/0x20 [ 2180.237633] [ 2180.238879] ? native_safe_halt+0x2e/0x50 [ 2180.242955] kvm_wait+0xcd/0xe0 [ 2180.245308] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2180.247084] do_raw_spin_lock+0x177/0x190 [ 2180.254789] _raw_spin_lock+0x21/0x30 [ 2180.259862] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2180.268729] ? do_raw_spin_unlock+0x75/0x190 [ 2180.272733] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2180.279767] ? do_wait_intr+0xf0/0xf0 [ 2180.283895] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2180.293979] mdc_reint+0x2d/0x110 [mdc] [ 2180.304754] mdc_unlink+0x658/0x870 [mdc] [ 2180.309583] lmv_unlink+0x34d/0xbd0 [lmv] [ 2180.312265] ll_unlink+0x24d/0x7f0 [lustre] [ 2180.314787] vfs_unlink+0x202/0x330 [ 2180.315978] do_unlinkat+0x2fa/0x460 [ 2180.317735] __x64_sys_unlink+0x37/0x50 [ 2180.319479] do_syscall_64+0xc1/0x440 [ 2180.321237] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2180.323476] RIP: 0033:0x7f33957fc10b [ 2180.324970] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2180.332515] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2180.334967] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2180.337072] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2180.339862] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2180.341962] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2180.344519] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2184.053075] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2184.058167] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2184.060175] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2184.084601] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2184.084621] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2184.084625] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2184.084644] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2184.084648] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2184.084656] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2184.084659] RDX: 0000000000003bed RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2184.084661] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2184.084663] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2184.084665] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2184.084669] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2184.084672] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2184.084674] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2184.084683] Call Trace: [ 2184.084690] [ 2184.084699] ? show_regs.cold.9+0x22/0x2f [ 2184.084712] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2184.084718] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2184.084723] ? watchdog+0x40/0x40 [ 2184.084728] ? hrtimer_interrupt+0x128/0x2c0 [ 2184.084732] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2184.084745] ? apic_timer_interrupt+0xf/0x20 [ 2184.084749] [ 2184.084753] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2184.084758] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2184.084774] do_raw_spin_lock+0x177/0x190 [ 2184.084780] _raw_spin_lock+0x21/0x30 [ 2184.084786] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2184.085237] obd_mod_free+0x46/0x140 [ptlrpc] [ 2184.085944] mdc_commit_open+0x91/0xb0 [mdc] [ 2184.085996] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2184.086215] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2184.086887] after_reply+0xe2c/0x1780 [ptlrpc] [ 2184.087140] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2184.087359] ? __next_timer_interrupt+0x160/0x160 [ 2184.087372] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2184.087894] ? do_wait_intr+0xf0/0xf0 [ 2184.087905] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2184.088202] kthread+0x1d1/0x200 [ 2184.088209] ? set_kthread_struct+0x70/0x70 [ 2184.088213] ret_from_fork+0x1f/0x30 [ 2184.284436] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2184.317211] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2184.331549] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2184.339528] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2184.353450] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2184.361611] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2184.377093] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2184.382840] RAX: 0000000000005152 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2184.388605] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2184.392321] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2184.401217] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2184.411126] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2184.419713] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2184.426907] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2184.429348] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2184.442222] Call Trace: [ 2184.443113] [ 2184.445815] ? show_regs.cold.9+0x22/0x2f [ 2184.449967] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2184.453107] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2184.460060] ? watchdog+0x40/0x40 [ 2184.460991] ? hrtimer_interrupt+0x128/0x2c0 [ 2184.462337] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2184.468205] ? apic_timer_interrupt+0xf/0x20 [ 2184.471796] [ 2184.472828] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2184.476544] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2184.479938] do_raw_spin_lock+0x177/0x190 [ 2184.484625] _raw_spin_lock+0x21/0x30 [ 2184.494993] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2184.499540] process_one_work+0x2c8/0x700 [ 2184.502070] worker_thread+0x296/0x6e0 [ 2184.504605] ? rescuer_thread+0x570/0x570 [ 2184.506645] kthread+0x1d1/0x200 [ 2184.508797] ? set_kthread_struct+0x70/0x70 [ 2184.510842] ret_from_fork+0x1f/0x30 [ 2208.060061] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2208.066702] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2208.115838] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2208.123676] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2208.128741] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2208.131737] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2208.151830] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2208.155541] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2208.160203] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2208.163701] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2208.168266] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2208.175818] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2208.181381] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2208.187360] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2208.192688] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2208.205241] Call Trace: [ 2208.207834] [ 2208.209585] ? show_regs.cold.9+0x22/0x2f [ 2208.211807] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2208.214767] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2208.219760] ? watchdog+0x40/0x40 [ 2208.220992] ? hrtimer_interrupt+0x128/0x2c0 [ 2208.224550] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2208.230019] ? apic_timer_interrupt+0xf/0x20 [ 2208.233466] [ 2208.234260] ? native_safe_halt+0x2e/0x50 [ 2208.242215] kvm_wait+0xcd/0xe0 [ 2208.245689] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2208.248488] do_raw_spin_lock+0x177/0x190 [ 2208.253569] _raw_spin_lock+0x21/0x30 [ 2208.259650] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2208.266303] ? do_raw_spin_unlock+0x75/0x190 [ 2208.272222] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2208.278554] ? do_wait_intr+0xf0/0xf0 [ 2208.281826] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2208.286556] mdc_reint+0x2d/0x110 [mdc] [ 2208.288015] mdc_unlink+0x658/0x870 [mdc] [ 2208.292187] lmv_unlink+0x34d/0xbd0 [lmv] [ 2208.295692] ll_unlink+0x24d/0x7f0 [lustre] [ 2208.299342] vfs_unlink+0x202/0x330 [ 2208.300456] do_unlinkat+0x2fa/0x460 [ 2208.303174] __x64_sys_unlink+0x37/0x50 [ 2208.306656] do_syscall_64+0xc1/0x440 [ 2208.312170] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2208.315327] RIP: 0033:0x7f33957fc10b [ 2208.317720] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2208.333946] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2208.343596] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2208.348623] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2208.352097] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2208.357653] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2208.364299] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2212.059256] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2212.061370] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2212.070843] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2212.070923] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2212.070932] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2212.070939] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2212.071256] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2212.071280] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2212.071285] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2212.071290] RAX: 0000000000007a08 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2212.071292] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2212.071294] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2212.071297] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2212.071299] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2212.071302] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2212.071305] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2212.071307] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2212.071315] Call Trace: [ 2212.071321] [ 2212.071327] ? show_regs.cold.9+0x22/0x2f [ 2212.071338] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2212.071345] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2212.071349] ? watchdog+0x40/0x40 [ 2212.071354] ? hrtimer_interrupt+0x128/0x2c0 [ 2212.071358] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2212.071368] ? apic_timer_interrupt+0xf/0x20 [ 2212.071372] [ 2212.071375] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2212.071380] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2212.071384] do_raw_spin_lock+0x177/0x190 [ 2212.071389] _raw_spin_lock+0x21/0x30 [ 2212.071393] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2212.071611] process_one_work+0x2c8/0x700 [ 2212.071618] worker_thread+0x296/0x6e0 [ 2212.071623] ? rescuer_thread+0x570/0x570 [ 2212.071626] kthread+0x1d1/0x200 [ 2212.071631] ? set_kthread_struct+0x70/0x70 [ 2212.071634] ret_from_fork+0x1f/0x30 [ 2212.318180] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2212.385313] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2212.392127] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2212.395899] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2212.399098] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2212.431607] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2212.448755] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2212.461995] RDX: 00000000000029dc RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2212.473277] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2212.480324] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2212.487118] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2212.495222] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2212.503306] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2212.506553] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2212.513666] Call Trace: [ 2212.516245] [ 2212.518482] ? show_regs.cold.9+0x22/0x2f [ 2212.521367] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2212.523862] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2212.526094] ? watchdog+0x40/0x40 [ 2212.528123] ? hrtimer_interrupt+0x128/0x2c0 [ 2212.530961] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2212.533497] ? apic_timer_interrupt+0xf/0x20 [ 2212.536040] [ 2212.537269] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2212.541935] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2212.545188] do_raw_spin_lock+0x177/0x190 [ 2212.547982] _raw_spin_lock+0x21/0x30 [ 2212.550252] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2212.553500] obd_mod_free+0x46/0x140 [ptlrpc] [ 2212.556675] mdc_commit_open+0x91/0xb0 [mdc] [ 2212.559602] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2212.564196] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2212.570385] after_reply+0xe2c/0x1780 [ptlrpc] [ 2212.576647] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2212.584026] ? __next_timer_interrupt+0x160/0x160 [ 2212.594018] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2212.603721] ? do_wait_intr+0xf0/0xf0 [ 2212.606455] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2212.612188] kthread+0x1d1/0x200 [ 2212.617411] ? set_kthread_struct+0x70/0x70 [ 2212.621479] ret_from_fork+0x1f/0x30 [ 2212.915119] INFO: task writemany:37218 blocked for more than 120 seconds. [ 2212.919064] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2212.924202] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2212.931060] task:writemany state:D stack:0 pid:37218 ppid:37213 flags:0x80000004 [ 2212.934739] Call Trace: [ 2212.935666] __schedule+0x351/0xcb0 [ 2212.940414] schedule+0xc0/0x180 [ 2212.944824] schedule_preempt_disabled+0x21/0x40 [ 2212.946209] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2212.948514] ? __might_sleep+0x59/0xc0 [ 2212.951020] down_write+0x80/0xd0 [ 2212.954840] do_unlinkat+0x184/0x460 [ 2212.958743] __x64_sys_unlink+0x37/0x50 [ 2212.961404] do_syscall_64+0xc1/0x440 [ 2212.964145] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2212.967507] RIP: 0033:0x7f33957fc10b [ 2212.970958] Code: Unable to access opcode bytes at RIP 0x7f33957fc0e1. [ 2212.975083] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2212.981534] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f33957fc10b [ 2212.984587] RDX: 0000000000000000 RSI: 00007ffe42a47a64 RDI: 00007ffe42a47ae0 [ 2212.987545] RBP: 00007ffe42a47ee0 R08: 00007f3395b9425c R09: 00007f3395b94260 [ 2212.991042] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2212.994105] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2212.997501] INFO: task writemany:37220 blocked for more than 120 seconds. [ 2213.000035] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2213.002818] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2213.005821] task:writemany state:D stack:0 pid:37220 ppid:37213 flags:0x80000004 [ 2213.008875] Call Trace: [ 2213.010140] __schedule+0x351/0xcb0 [ 2213.011694] schedule+0xc0/0x180 [ 2213.014096] schedule_preempt_disabled+0x21/0x40 [ 2213.017128] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2213.019341] ? __might_sleep+0x59/0xc0 [ 2213.021106] down_write+0x80/0xd0 [ 2213.023225] do_unlinkat+0x184/0x460 [ 2213.026623] __x64_sys_unlink+0x37/0x50 [ 2213.030916] do_syscall_64+0xc1/0x440 [ 2213.034086] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2213.040416] RIP: 0033:0x7f33957fc10b [ 2213.041635] Code: Unable to access opcode bytes at RIP 0x7f33957fc0e1. [ 2213.045949] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2213.050575] RAX: ffffffffffffffda RBX: 000000000011a178 RCX: 00007f33957fc10b [ 2213.053651] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2213.057666] RBP: 00007ffe42a47ee0 R08: 00007f3395b94230 R09: 00007f3395b94260 [ 2213.062983] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2213.065166] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2213.109172] INFO: task writemany:37221 blocked for more than 120 seconds. [ 2213.112836] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2213.119111] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2213.126375] task:writemany state:D stack:0 pid:37221 ppid:37213 flags:0x80000004 [ 2213.131081] Call Trace: [ 2213.132561] __schedule+0x351/0xcb0 [ 2213.134402] schedule+0xc0/0x180 [ 2213.136160] schedule_preempt_disabled+0x21/0x40 [ 2213.138304] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2213.142983] ? __might_sleep+0x59/0xc0 [ 2213.146791] down_write+0x80/0xd0 [ 2213.150009] do_unlinkat+0x184/0x460 [ 2213.153115] __x64_sys_unlink+0x37/0x50 [ 2213.158107] do_syscall_64+0xc1/0x440 [ 2213.161641] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2213.165922] RIP: 0033:0x7f33957fc10b [ 2213.167954] Code: Unable to access opcode bytes at RIP 0x7f33957fc0e1. [ 2213.170819] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2213.174558] RAX: ffffffffffffffda RBX: 00000000006789b8 RCX: 00007f33957fc10b [ 2213.177209] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2213.180172] RBP: 00007ffe42a47ee0 R08: 00007f3395b9423c R09: 00007f3395b94260 [ 2213.183698] R10: 00007ffe42a47823 R11: 0000000000000246 R12: 0000000000000003 [ 2213.186886] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2213.189246] INFO: task writemany:37222 blocked for more than 120 seconds. [ 2213.191502] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2213.195338] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2213.199385] task:writemany state:D stack:0 pid:37222 ppid:37213 flags:0x80000004 [ 2213.203971] Call Trace: [ 2213.210550] __schedule+0x351/0xcb0 [ 2213.211849] schedule+0xc0/0x180 [ 2213.215859] schedule_preempt_disabled+0x21/0x40 [ 2213.218037] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2213.220912] ? __might_sleep+0x59/0xc0 [ 2213.222702] down_write+0x80/0xd0 [ 2213.224394] do_unlinkat+0x184/0x460 [ 2213.226028] __x64_sys_unlink+0x37/0x50 [ 2213.227874] do_syscall_64+0xc1/0x440 [ 2213.232712] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2213.235695] RIP: 0033:0x7f33957fc10b [ 2213.239530] Code: Unable to access opcode bytes at RIP 0x7f33957fc0e1. [ 2213.243620] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2213.250663] RAX: ffffffffffffffda RBX: 000000000102a848 RCX: 00007f33957fc10b [ 2213.255967] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2213.258059] RBP: 00007ffe42a47ee0 R08: 00007f3395b94254 R09: 00007f3395b94260 [ 2213.260214] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2213.262439] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2236.060001] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2236.078124] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2236.131798] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2236.144886] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2236.150247] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2236.153684] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2236.167619] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2236.174488] RAX: 0000000000000e82 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2236.185271] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2236.194271] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2236.203947] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2236.214978] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2236.224748] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2236.232183] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2236.238132] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2236.245254] Call Trace: [ 2236.246723] [ 2236.247250] ? show_regs.cold.9+0x22/0x2f [ 2236.250645] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2236.254027] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2236.257072] ? watchdog+0x40/0x40 [ 2236.259131] ? hrtimer_interrupt+0x128/0x2c0 [ 2236.262898] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2236.270099] ? apic_timer_interrupt+0xf/0x20 [ 2236.273400] [ 2236.275848] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2236.279926] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2236.281623] do_raw_spin_lock+0x177/0x190 [ 2236.284915] _raw_spin_lock+0x21/0x30 [ 2236.286889] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2236.290831] ? do_raw_spin_unlock+0x75/0x190 [ 2236.296080] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2236.301155] ? do_wait_intr+0xf0/0xf0 [ 2236.302697] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2236.310158] mdc_reint+0x2d/0x110 [mdc] [ 2236.312981] mdc_unlink+0x658/0x870 [mdc] [ 2236.315436] lmv_unlink+0x34d/0xbd0 [lmv] [ 2236.318228] ll_unlink+0x24d/0x7f0 [lustre] [ 2236.321747] vfs_unlink+0x202/0x330 [ 2236.326957] do_unlinkat+0x2fa/0x460 [ 2236.333860] __x64_sys_unlink+0x37/0x50 [ 2236.335660] do_syscall_64+0xc1/0x440 [ 2236.342808] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2236.348848] RIP: 0033:0x7f33957fc10b [ 2236.351681] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2236.379315] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2236.384963] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2236.387273] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2236.403508] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2236.412551] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2236.424667] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2240.053355] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2240.057425] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2240.058532] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2240.080418] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2240.080432] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2240.080435] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2240.080451] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2240.080455] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2240.080462] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2240.080464] RDX: 0000000000000d48 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2240.080466] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2240.080469] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2240.080471] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2240.080474] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2240.080476] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2240.080479] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2240.080488] Call Trace: [ 2240.080493] [ 2240.080500] ? show_regs.cold.9+0x22/0x2f [ 2240.080509] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2240.080515] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2240.080520] ? watchdog+0x40/0x40 [ 2240.080524] ? hrtimer_interrupt+0x128/0x2c0 [ 2240.080528] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2240.080539] ? apic_timer_interrupt+0xf/0x20 [ 2240.080544] [ 2240.080547] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2240.080552] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2240.080556] do_raw_spin_lock+0x177/0x190 [ 2240.080561] _raw_spin_lock+0x21/0x30 [ 2240.080568] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2240.080876] obd_mod_free+0x46/0x140 [ptlrpc] [ 2240.081408] mdc_commit_open+0x91/0xb0 [mdc] [ 2240.081455] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2240.081672] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2240.081875] after_reply+0xe2c/0x1780 [ptlrpc] [ 2240.089102] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix [ 2240.094940] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2240.095442] ? __next_timer_interrupt+0x160/0x160 [ 2240.095454] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2240.095668] ? do_wait_intr+0xf0/0xf0 [ 2240.095677] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2240.095896] kthread+0x1d1/0x200 [ 2240.095902] ? set_kthread_struct+0x70/0x70 [ 2240.095906] ret_from_fork+0x1f/0x30 [ 2240.326776] serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2240.334835] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2240.342490] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2240.348177] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2240.359110] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2240.363386] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2240.381725] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2240.385926] RAX: 0000000000004a8f RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2240.394347] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2240.398285] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2240.401507] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2240.404025] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2240.406362] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2240.409779] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2240.413579] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2240.417812] Call Trace: [ 2240.418768] [ 2240.419708] ? show_regs.cold.9+0x22/0x2f [ 2240.421166] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2240.423355] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2240.425375] ? watchdog+0x40/0x40 [ 2240.426980] ? hrtimer_interrupt+0x128/0x2c0 [ 2240.429564] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2240.431850] ? apic_timer_interrupt+0xf/0x20 [ 2240.434036] [ 2240.434963] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2240.437149] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2240.440363] do_raw_spin_lock+0x177/0x190 [ 2240.445282] _raw_spin_lock+0x21/0x30 [ 2240.447199] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2240.452209] process_one_work+0x2c8/0x700 [ 2240.455602] worker_thread+0x296/0x6e0 [ 2240.459860] ? rescuer_thread+0x570/0x570 [ 2240.462366] kthread+0x1d1/0x200 [ 2240.467146] ? set_kthread_struct+0x70/0x70 [ 2240.473735] ret_from_fork+0x1f/0x30 [ 2264.060066] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2264.073286] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2264.141321] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2264.153922] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2264.160375] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2264.162981] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2264.174015] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2264.178925] RAX: 000000000000634f RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2264.184703] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2264.192799] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2264.195677] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2264.200629] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2264.202933] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2264.206680] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2264.208403] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2264.213149] Call Trace: [ 2264.214156] [ 2264.215839] ? show_regs.cold.9+0x22/0x2f [ 2264.217448] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2264.223564] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2264.229717] ? watchdog+0x40/0x40 [ 2264.232700] ? hrtimer_interrupt+0x128/0x2c0 [ 2264.242091] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2264.246195] ? apic_timer_interrupt+0xf/0x20 [ 2264.249191] [ 2264.251264] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2264.255969] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2264.261676] do_raw_spin_lock+0x177/0x190 [ 2264.264218] _raw_spin_lock+0x21/0x30 [ 2264.274088] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2264.280426] ? do_raw_spin_unlock+0x75/0x190 [ 2264.292475] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2264.296589] ? do_wait_intr+0xf0/0xf0 [ 2264.300207] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2264.308778] mdc_reint+0x2d/0x110 [mdc] [ 2264.312648] mdc_unlink+0x658/0x870 [mdc] [ 2264.316722] lmv_unlink+0x34d/0xbd0 [lmv] [ 2264.322512] ll_unlink+0x24d/0x7f0 [lustre] [ 2264.324566] vfs_unlink+0x202/0x330 [ 2264.327103] do_unlinkat+0x2fa/0x460 [ 2264.336440] __x64_sys_unlink+0x37/0x50 [ 2264.340431] do_syscall_64+0xc1/0x440 [ 2264.347684] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2264.351692] RIP: 0033:0x7f33957fc10b [ 2264.356215] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2264.385914] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2264.394997] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2264.402795] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2264.407701] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2264.416494] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2264.422340] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2268.057944] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2268.058028] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2268.065106] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) [ 2268.072890] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2268.072994] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2268.073003] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2268.073013] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2268.075266] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2268.075284] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2268.075289] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2268.075293] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2268.075296] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2268.075298] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2268.075300] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2268.075303] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2268.075306] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2268.075309] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2268.075312] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2268.075837] Call Trace: [ 2268.075844] [ 2268.075849] ? show_regs.cold.9+0x22/0x2f [ 2268.075861] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2268.075868] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2268.075873] ? watchdog+0x40/0x40 [ 2268.075877] ? hrtimer_interrupt+0x128/0x2c0 [ 2268.075882] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2268.075889] ? apic_timer_interrupt+0xf/0x20 [ 2268.075893] [ 2268.075896] ? native_safe_halt+0x2e/0x50 [ 2268.075900] kvm_wait+0xcd/0xe0 [ 2268.075909] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2268.075918] do_raw_spin_lock+0x177/0x190 [ 2268.075923] _raw_spin_lock+0x21/0x30 [ 2268.075928] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2268.077190] process_one_work+0x2c8/0x700 [ 2268.077202] worker_thread+0x296/0x6e0 [ 2268.077207] ? rescuer_thread+0x570/0x570 [ 2268.077210] kthread+0x1d1/0x200 [ 2268.077214] ? set_kthread_struct+0x70/0x70 [ 2268.077218] ret_from_fork+0x1f/0x30 [ 2268.438918] ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2268.472404] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2268.486465] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2268.507107] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2268.528352] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2268.547106] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2268.555685] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2268.571270] RDX: 0000000000002b38 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2268.580927] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2268.589558] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2268.603302] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2268.617276] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2268.633979] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2268.642859] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2268.654572] Call Trace: [ 2268.657048] [ 2268.657875] ? show_regs.cold.9+0x22/0x2f [ 2268.661227] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2268.663717] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2268.669035] ? watchdog+0x40/0x40 [ 2268.674280] ? hrtimer_interrupt+0x128/0x2c0 [ 2268.677104] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2268.680005] ? apic_timer_interrupt+0xf/0x20 [ 2268.687143] [ 2268.691642] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2268.695967] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2268.701971] do_raw_spin_lock+0x177/0x190 [ 2268.704115] _raw_spin_lock+0x21/0x30 [ 2268.707146] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2268.716539] obd_mod_free+0x46/0x140 [ptlrpc] [ 2268.723556] mdc_commit_open+0x91/0xb0 [mdc] [ 2268.731798] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2268.743072] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2268.757981] after_reply+0xe2c/0x1780 [ptlrpc] [ 2268.760821] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2268.769464] ? __next_timer_interrupt+0x160/0x160 [ 2268.774626] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2268.778048] ? do_wait_intr+0xf0/0xf0 [ 2268.783022] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2268.786989] kthread+0x1d1/0x200 [ 2268.789577] ? set_kthread_struct+0x70/0x70 [ 2268.793699] ret_from_fork+0x1f/0x30 [ 2292.067498] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2292.078064] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2292.114021] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2292.123530] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2292.125904] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2292.132154] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2292.144686] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2292.147537] RAX: 0000000000007796 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2292.149960] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2292.152494] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2292.154725] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2292.156836] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2292.159278] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2292.161558] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2292.163351] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2292.165482] Call Trace: [ 2292.166149] [ 2292.166833] ? show_regs.cold.9+0x22/0x2f [ 2292.168404] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2292.170723] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2292.173979] ? watchdog+0x40/0x40 [ 2292.177157] ? hrtimer_interrupt+0x128/0x2c0 [ 2292.181471] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2292.184075] ? apic_timer_interrupt+0xf/0x20 [ 2292.188818] [ 2292.191610] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2292.196838] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2292.199388] do_raw_spin_lock+0x177/0x190 [ 2292.202744] _raw_spin_lock+0x21/0x30 [ 2292.207325] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2292.211069] ? do_raw_spin_unlock+0x75/0x190 [ 2292.215336] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2292.222840] ? do_wait_intr+0xf0/0xf0 [ 2292.224299] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2292.227592] mdc_reint+0x2d/0x110 [mdc] [ 2292.229645] mdc_unlink+0x658/0x870 [mdc] [ 2292.232677] lmv_unlink+0x34d/0xbd0 [lmv] [ 2292.236814] ll_unlink+0x24d/0x7f0 [lustre] [ 2292.240284] vfs_unlink+0x202/0x330 [ 2292.244527] do_unlinkat+0x2fa/0x460 [ 2292.246161] __x64_sys_unlink+0x37/0x50 [ 2292.248373] do_syscall_64+0xc1/0x440 [ 2292.252265] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2292.256933] RIP: 0033:0x7f33957fc10b [ 2292.260431] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2292.278293] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2292.284674] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2292.291843] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2292.295356] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2292.310381] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2292.317547] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2296.056209] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2296.060085] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common [ 2296.060749] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2296.060805] sb_edac [ 2296.080695] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2296.080779] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2296.080787] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2296.080795] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2296.081219] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2296.081231] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2296.081235] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2296.081240] RAX: 0000000000000394 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2296.081242] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2296.081245] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2296.081247] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2296.081249] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2296.081252] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2296.081419] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2296.081422] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2296.081430] Call Trace: [ 2296.081436] [ 2296.081442] ? show_regs.cold.9+0x22/0x2f [ 2296.081452] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2296.081459] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2296.081464] ? watchdog+0x40/0x40 [ 2296.081467] ? hrtimer_interrupt+0x128/0x2c0 [ 2296.081471] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2296.081479] ? apic_timer_interrupt+0xf/0x20 [ 2296.081484] [ 2296.081486] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2296.081491] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2296.081497] do_raw_spin_lock+0x177/0x190 [ 2296.081501] _raw_spin_lock+0x21/0x30 [ 2296.081507] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2296.081739] process_one_work+0x2c8/0x700 [ 2296.081746] worker_thread+0x296/0x6e0 [ 2296.081750] ? rescuer_thread+0x570/0x570 [ 2296.081754] kthread+0x1d1/0x200 [ 2296.081758] ? set_kthread_struct+0x70/0x70 [ 2296.081762] ret_from_fork+0x1f/0x30 [ 2296.286149] rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2296.307243] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2296.315593] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2296.324780] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2296.328939] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2296.351647] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2296.355365] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2296.359311] RDX: 00000000000036a3 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2296.372559] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2296.380685] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2296.386570] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2296.392527] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2296.402959] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2296.407747] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2296.412972] Call Trace: [ 2296.420525] [ 2296.422917] ? show_regs.cold.9+0x22/0x2f [ 2296.424967] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2296.427313] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2296.429429] ? watchdog+0x40/0x40 [ 2296.430606] ? hrtimer_interrupt+0x128/0x2c0 [ 2296.435024] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2296.441727] ? apic_timer_interrupt+0xf/0x20 [ 2296.450735] [ 2296.455499] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2296.465180] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2296.474521] do_raw_spin_lock+0x177/0x190 [ 2296.482716] _raw_spin_lock+0x21/0x30 [ 2296.489667] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2296.499740] obd_mod_free+0x46/0x140 [ptlrpc] [ 2296.511667] mdc_commit_open+0x91/0xb0 [mdc] [ 2296.518252] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2296.523409] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2296.527527] after_reply+0xe2c/0x1780 [ptlrpc] [ 2296.535607] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2296.541432] ? __next_timer_interrupt+0x160/0x160 [ 2296.543801] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2296.546801] ? do_wait_intr+0xf0/0xf0 [ 2296.552448] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2296.565486] kthread+0x1d1/0x200 [ 2296.570715] ? set_kthread_struct+0x70/0x70 [ 2296.578115] ret_from_fork+0x1f/0x30 [ 2320.060157] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2320.069121] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2320.114448] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2320.125882] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2320.135423] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2320.145726] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2320.163370] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2320.171155] RAX: 0000000000005f6d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2320.180597] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2320.193264] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2320.200147] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2320.206124] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2320.209943] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2320.214178] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2320.218815] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2320.223937] Call Trace: [ 2320.228977] [ 2320.230183] ? show_regs.cold.9+0x22/0x2f [ 2320.232726] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2320.242755] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2320.250216] ? watchdog+0x40/0x40 [ 2320.255151] ? hrtimer_interrupt+0x128/0x2c0 [ 2320.257059] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2320.260975] ? apic_timer_interrupt+0xf/0x20 [ 2320.263723] [ 2320.265414] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2320.271026] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2320.275321] do_raw_spin_lock+0x177/0x190 [ 2320.283000] _raw_spin_lock+0x21/0x30 [ 2320.286954] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2320.294740] ? do_raw_spin_unlock+0x75/0x190 [ 2320.302321] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2320.311358] ? do_wait_intr+0xf0/0xf0 [ 2320.313910] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2320.322000] mdc_reint+0x2d/0x110 [mdc] [ 2320.327505] mdc_unlink+0x658/0x870 [mdc] [ 2320.330891] lmv_unlink+0x34d/0xbd0 [lmv] [ 2320.333327] ll_unlink+0x24d/0x7f0 [lustre] [ 2320.341972] vfs_unlink+0x202/0x330 [ 2320.346316] do_unlinkat+0x2fa/0x460 [ 2320.348668] __x64_sys_unlink+0x37/0x50 [ 2320.350867] do_syscall_64+0xc1/0x440 [ 2320.353878] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2320.356636] RIP: 0033:0x7f33957fc10b [ 2320.358815] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2320.368781] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2320.374711] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2320.379049] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2320.383124] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2320.387438] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2320.393639] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2324.053023] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2324.060410] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2324.065292] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2324.065390] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2324.065399] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2324.065402] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2324.065419] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2324.065424] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2324.065428] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2324.065430] RDX: 000000000000064a RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2324.065433] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2324.065434] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2324.065436] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2324.065440] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2324.065443] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2324.065445] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2324.065454] Call Trace: [ 2324.065461] [ 2324.065468] ? show_regs.cold.9+0x22/0x2f [ 2324.065478] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2324.065485] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2324.065489] ? watchdog+0x40/0x40 [ 2324.065493] ? hrtimer_interrupt+0x128/0x2c0 [ 2324.065497] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2324.065505] ? apic_timer_interrupt+0xf/0x20 [ 2324.065509] [ 2324.065512] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2324.065517] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2324.065522] do_raw_spin_lock+0x177/0x190 [ 2324.065526] _raw_spin_lock+0x21/0x30 [ 2324.065532] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2324.068768] obd_mod_free+0x46/0x140 [ptlrpc] [ 2324.069906] mdc_commit_open+0x91/0xb0 [mdc] [ 2324.069945] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2324.070511] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2324.080373] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2324.080456] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2324.080464] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2324.080471] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2324.080757] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2324.080771] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2324.080775] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2324.080779] RAX: 0000000000002711 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2324.080782] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2324.080784] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2324.080786] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2324.080788] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2324.080791] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2324.080794] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2324.080797] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2324.080807] Call Trace: [ 2324.080812] [ 2324.080818] ? show_regs.cold.9+0x22/0x2f [ 2324.080829] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2324.080836] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2324.080840] ? watchdog+0x40/0x40 [ 2324.080843] ? hrtimer_interrupt+0x128/0x2c0 [ 2324.080847] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2324.080855] ? apic_timer_interrupt+0xf/0x20 [ 2324.080859] [ 2324.080862] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2324.080866] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2324.080870] do_raw_spin_lock+0x177/0x190 [ 2324.080874] _raw_spin_lock+0x21/0x30 [ 2324.080879] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2324.081092] process_one_work+0x2c8/0x700 [ 2324.081099] worker_thread+0x296/0x6e0 [ 2324.081104] ? rescuer_thread+0x570/0x570 [ 2324.081107] kthread+0x1d1/0x200 [ 2324.081111] ? set_kthread_struct+0x70/0x70 [ 2324.081115] ret_from_fork+0x1f/0x30 [ 2324.588490] after_reply+0xe2c/0x1780 [ptlrpc] [ 2324.594651] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2324.597140] ? __next_timer_interrupt+0x160/0x160 [ 2324.604291] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2324.606164] ? do_wait_intr+0xf0/0xf0 [ 2324.607819] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2324.610512] kthread+0x1d1/0x200 [ 2324.611586] ? set_kthread_struct+0x70/0x70 [ 2324.612639] ret_from_fork+0x1f/0x30 [ 2335.713155] INFO: task khugepaged:48 blocked for more than 120 seconds. [ 2335.717627] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2335.719663] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2335.729845] task:khugepaged state:D stack:0 pid:48 ppid:2 flags:0x80004000 [ 2335.740790] Call Trace: [ 2335.742144] __schedule+0x351/0xcb0 [ 2335.745433] ? wait_for_completion+0xae/0x1e0 [ 2335.746912] schedule+0xc0/0x180 [ 2335.749038] schedule_timeout+0x126/0x190 [ 2335.751502] ? try_to_wake_up+0x80/0x710 [ 2335.753523] ? __prepare_to_swait+0x5b/0x90 [ 2335.763029] ? do_raw_spin_unlock+0x75/0x190 [ 2335.767052] wait_for_completion+0xf0/0x1e0 [ 2335.770274] flush_work+0x1f3/0x390 [ 2335.777028] ? flush_workqueue_prep_pwqs+0x200/0x200 [ 2335.783581] ? do_raw_spin_unlock+0x75/0x190 [ 2335.787235] ? __pagevec_release+0x50/0x50 [ 2335.789292] lru_add_drain_all+0x1d4/0x280 [ 2335.794577] khugepaged+0xa2/0x16f0 [ 2335.798539] ? woken_wake_function+0x30/0x30 [ 2335.803598] ? collapse_pte_mapped_thp+0x8a0/0x8a0 [ 2335.813639] kthread+0x1d1/0x200 [ 2335.817733] ? set_kthread_struct+0x70/0x70 [ 2335.824854] ret_from_fork+0x1f/0x30 [ 2335.838520] INFO: task writemany:37218 blocked for more than 120 seconds. [ 2335.855786] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2335.870183] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2335.880826] task:writemany state:D stack:0 pid:37218 ppid:37213 flags:0x80000004 [ 2335.894295] Call Trace: [ 2335.895158] __schedule+0x351/0xcb0 [ 2335.899862] schedule+0xc0/0x180 [ 2335.901033] schedule_preempt_disabled+0x21/0x40 [ 2335.908196] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2335.913920] ? __might_sleep+0x59/0xc0 [ 2335.918497] down_write+0x80/0xd0 [ 2335.919745] do_unlinkat+0x184/0x460 [ 2335.923058] __x64_sys_unlink+0x37/0x50 [ 2335.925434] do_syscall_64+0xc1/0x440 [ 2335.927743] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2335.930762] RIP: 0033:0x7f33957fc10b [ 2335.934452] Code: Unable to access opcode bytes at RIP 0x7f33957fc0e1. [ 2335.943284] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2335.949366] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f33957fc10b [ 2335.953645] RDX: 0000000000000000 RSI: 00007ffe42a47a64 RDI: 00007ffe42a47ae0 [ 2335.966654] RBP: 00007ffe42a47ee0 R08: 00007f3395b9425c R09: 00007f3395b94260 [ 2335.972184] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2335.976210] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2335.983114] INFO: task writemany:37220 blocked for more than 120 seconds. [ 2335.986847] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2335.991347] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2336.006704] task:writemany state:D stack:0 pid:37220 ppid:37213 flags:0x80000004 [ 2336.010741] Call Trace: [ 2336.015827] __schedule+0x351/0xcb0 [ 2336.018125] schedule+0xc0/0x180 [ 2336.025786] schedule_preempt_disabled+0x21/0x40 [ 2336.034296] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2336.037272] ? __might_sleep+0x59/0xc0 [ 2336.040178] down_write+0x80/0xd0 [ 2336.042504] do_unlinkat+0x184/0x460 [ 2336.043967] __x64_sys_unlink+0x37/0x50 [ 2336.050262] do_syscall_64+0xc1/0x440 [ 2336.052128] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2336.059472] RIP: 0033:0x7f33957fc10b [ 2336.061384] Code: Unable to access opcode bytes at RIP 0x7f33957fc0e1. [ 2336.069991] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2336.077957] RAX: ffffffffffffffda RBX: 000000000011a178 RCX: 00007f33957fc10b [ 2336.084603] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2336.092274] RBP: 00007ffe42a47ee0 R08: 00007f3395b94230 R09: 00007f3395b94260 [ 2336.097360] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2336.102327] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2336.127181] INFO: task writemany:37221 blocked for more than 120 seconds. [ 2336.134264] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2336.139412] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2336.146473] task:writemany state:D stack:0 pid:37221 ppid:37213 flags:0x80000004 [ 2336.155524] Call Trace: [ 2336.159525] __schedule+0x351/0xcb0 [ 2336.169418] schedule+0xc0/0x180 [ 2336.172599] schedule_preempt_disabled+0x21/0x40 [ 2336.175866] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2336.182777] ? __might_sleep+0x59/0xc0 [ 2336.185036] down_write+0x80/0xd0 [ 2336.190529] do_unlinkat+0x184/0x460 [ 2336.194646] __x64_sys_unlink+0x37/0x50 [ 2336.199130] do_syscall_64+0xc1/0x440 [ 2336.200412] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2336.211221] RIP: 0033:0x7f33957fc10b [ 2336.215961] Code: Unable to access opcode bytes at RIP 0x7f33957fc0e1. [ 2336.219391] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2336.225883] RAX: ffffffffffffffda RBX: 00000000006789b8 RCX: 00007f33957fc10b [ 2336.228141] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2336.234031] RBP: 00007ffe42a47ee0 R08: 00007f3395b9423c R09: 00007f3395b94260 [ 2336.241445] R10: 00007ffe42a47823 R11: 0000000000000246 R12: 0000000000000003 [ 2336.249188] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2336.264106] INFO: task writemany:37222 blocked for more than 120 seconds. [ 2336.268957] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2336.276455] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2336.280634] task:writemany state:D stack:0 pid:37222 ppid:37213 flags:0x80000004 [ 2336.284867] Call Trace: [ 2336.286525] __schedule+0x351/0xcb0 [ 2336.288823] schedule+0xc0/0x180 [ 2336.289725] schedule_preempt_disabled+0x21/0x40 [ 2336.292502] rwsem_down_write_slowpath+0x5d7/0xa40 [ 2336.296812] ? __might_sleep+0x59/0xc0 [ 2336.298902] down_write+0x80/0xd0 [ 2336.301465] do_unlinkat+0x184/0x460 [ 2336.304126] __x64_sys_unlink+0x37/0x50 [ 2336.307390] do_syscall_64+0xc1/0x440 [ 2336.309459] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2336.311568] RIP: 0033:0x7f33957fc10b [ 2336.312992] Code: Unable to access opcode bytes at RIP 0x7f33957fc0e1. [ 2336.324167] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2336.330898] RAX: ffffffffffffffda RBX: 000000000102a848 RCX: 00007f33957fc10b [ 2336.337791] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2336.345484] RBP: 00007ffe42a47ee0 R08: 00007f3395b94254 R09: 00007f3395b94260 [ 2336.353265] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2336.355236] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2338.023850] rcu: INFO: rcu_sched self-detected stall on CPU [ 2338.029774] rcu: 0-....: (131050 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=42856 [ 2338.039766] (t=240003 jiffies g=297885 q=3169) [ 2338.043663] NMI backtrace for cpu 0 [ 2338.046144] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2338.052916] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2338.059989] Call Trace: [ 2338.062572] [ 2338.064385] ? dump_stack+0xbb/0x10e [ 2338.066159] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 2338.071543] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 2338.073256] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 2338.077217] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 2338.080214] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 2338.086837] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 2338.088681] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 2338.098263] ? update_process_times+0x6d/0xc0 [ 2338.104140] ? tick_sched_handle.isra.18+0x27/0x90 [ 2338.112840] ? tick_sched_timer+0x53/0xb0 [ 2338.118222] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2338.129664] ? tick_sched_handle.isra.18+0x90/0x90 [ 2338.131456] ? hrtimer_interrupt+0x128/0x2c0 [ 2338.143832] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2338.151936] ? apic_timer_interrupt+0xf/0x20 [ 2338.157757] [ 2338.162502] ? native_safe_halt+0x2e/0x50 [ 2338.165985] ? kvm_wait+0xcd/0xe0 [ 2338.167641] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2338.173715] ? do_raw_spin_lock+0x177/0x190 [ 2338.178680] ? _raw_spin_lock+0x21/0x30 [ 2338.181851] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2338.189722] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2338.204646] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2338.212264] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2338.219004] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2338.222653] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2338.227994] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2338.232326] ? __next_timer_interrupt+0x160/0x160 [ 2338.236171] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2338.246047] ? do_wait_intr+0xf0/0xf0 [ 2338.250894] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2338.257335] ? kthread+0x1d1/0x200 [ 2338.262755] ? set_kthread_struct+0x70/0x70 [ 2338.268863] ? ret_from_fork+0x1f/0x30 [ 2348.061328] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2348.084766] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2348.129431] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2348.139269] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2348.147848] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2348.155242] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2348.179771] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2348.189612] RAX: 000000000000512d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2348.197614] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2348.207055] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2348.213637] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2348.221551] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2348.226984] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2348.232064] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2348.236595] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2348.242654] Call Trace: [ 2348.244818] [ 2348.246491] ? show_regs.cold.9+0x22/0x2f [ 2348.250468] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2348.255284] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2348.259060] ? watchdog+0x40/0x40 [ 2348.263645] ? hrtimer_interrupt+0x128/0x2c0 [ 2348.268805] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2348.272759] ? apic_timer_interrupt+0xf/0x20 [ 2348.278780] [ 2348.279885] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2348.285061] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2348.295776] do_raw_spin_lock+0x177/0x190 [ 2348.303105] _raw_spin_lock+0x21/0x30 [ 2348.315383] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2348.317696] ? do_raw_spin_unlock+0x75/0x190 [ 2348.322776] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2348.330753] ? do_wait_intr+0xf0/0xf0 [ 2348.337014] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2348.346262] mdc_reint+0x2d/0x110 [mdc] [ 2348.351945] mdc_unlink+0x658/0x870 [mdc] [ 2348.358060] lmv_unlink+0x34d/0xbd0 [lmv] [ 2348.365459] ll_unlink+0x24d/0x7f0 [lustre] [ 2348.373929] vfs_unlink+0x202/0x330 [ 2348.377967] do_unlinkat+0x2fa/0x460 [ 2348.386408] __x64_sys_unlink+0x37/0x50 [ 2348.394237] do_syscall_64+0xc1/0x440 [ 2348.399956] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2348.402275] RIP: 0033:0x7f33957fc10b [ 2348.410062] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2348.438048] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2348.450623] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2348.455812] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2348.473054] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2348.479852] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2348.489213] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2352.063727] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2352.076646] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2352.102270] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2352.111015] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2352.118986] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2352.125936] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2352.134501] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2352.162865] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2352.171537] RAX: 0000000000006289 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2352.176705] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2352.181204] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2352.192416] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2352.195978] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2352.207111] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2352.216401] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2352.218973] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2352.222417] Call Trace: [ 2352.224423] [ 2352.225790] ? show_regs.cold.9+0x22/0x2f [ 2352.229908] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2352.237435] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2352.243902] ? watchdog+0x40/0x40 [ 2352.245056] ? hrtimer_interrupt+0x128/0x2c0 [ 2352.248976] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2352.253617] ? apic_timer_interrupt+0xf/0x20 [ 2352.256773] [ 2352.259126] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2352.261660] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2352.265596] do_raw_spin_lock+0x177/0x190 [ 2352.269639] _raw_spin_lock+0x21/0x30 [ 2352.271405] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2352.274154] process_one_work+0x2c8/0x700 [ 2352.276069] worker_thread+0x296/0x6e0 [ 2352.280922] ? rescuer_thread+0x570/0x570 [ 2352.284561] kthread+0x1d1/0x200 [ 2352.286441] ? set_kthread_struct+0x70/0x70 [ 2352.288380] ret_from_fork+0x1f/0x30 [ 2364.053492] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2364.059994] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2364.107396] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2364.119236] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2364.129972] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2364.134681] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2364.153972] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2364.162133] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2364.172845] RDX: 00000000000063fe RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2364.184115] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2364.191402] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2364.200518] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2364.208675] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2364.219558] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2364.223164] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2364.229614] Call Trace: [ 2364.231873] [ 2364.233284] ? show_regs.cold.9+0x22/0x2f [ 2364.235207] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2364.238683] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2364.242397] ? watchdog+0x40/0x40 [ 2364.246963] ? hrtimer_interrupt+0x128/0x2c0 [ 2364.248619] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2364.259477] ? apic_timer_interrupt+0xf/0x20 [ 2364.263647] [ 2364.266569] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2364.274000] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2364.282467] do_raw_spin_lock+0x177/0x190 [ 2364.286708] _raw_spin_lock+0x21/0x30 [ 2364.288474] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2364.293115] obd_mod_free+0x46/0x140 [ptlrpc] [ 2364.298781] mdc_commit_open+0x91/0xb0 [mdc] [ 2364.301723] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2364.307710] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2364.309999] after_reply+0xe2c/0x1780 [ptlrpc] [ 2364.313745] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2364.316961] ? __next_timer_interrupt+0x160/0x160 [ 2364.319031] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2364.323072] ? do_wait_intr+0xf0/0xf0 [ 2364.325195] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2364.328387] kthread+0x1d1/0x200 [ 2364.329865] ? set_kthread_struct+0x70/0x70 [ 2364.332540] ret_from_fork+0x1f/0x30 [ 2376.065311] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2376.093986] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2376.146247] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2376.155305] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2376.161742] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2376.165859] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2376.189618] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2376.194788] RAX: 0000000000001cac RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2376.199304] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2376.203938] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2376.212271] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2376.222622] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2376.228219] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2376.233808] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2376.237487] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2376.243116] Call Trace: [ 2376.245284] [ 2376.246708] ? show_regs.cold.9+0x22/0x2f [ 2376.249878] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2376.253821] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2376.259490] ? watchdog+0x40/0x40 [ 2376.262742] ? hrtimer_interrupt+0x128/0x2c0 [ 2376.267627] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2376.272743] ? apic_timer_interrupt+0xf/0x20 [ 2376.276769] [ 2376.278841] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2376.286211] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2376.289705] do_raw_spin_lock+0x177/0x190 [ 2376.297283] _raw_spin_lock+0x21/0x30 [ 2376.301644] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2376.309066] ? do_raw_spin_unlock+0x75/0x190 [ 2376.325923] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2376.333473] ? do_wait_intr+0xf0/0xf0 [ 2376.334621] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2376.338780] mdc_reint+0x2d/0x110 [mdc] [ 2376.342269] mdc_unlink+0x658/0x870 [mdc] [ 2376.346420] lmv_unlink+0x34d/0xbd0 [lmv] [ 2376.349180] ll_unlink+0x24d/0x7f0 [lustre] [ 2376.351992] vfs_unlink+0x202/0x330 [ 2376.355237] do_unlinkat+0x2fa/0x460 [ 2376.357945] __x64_sys_unlink+0x37/0x50 [ 2376.359930] do_syscall_64+0xc1/0x440 [ 2376.362156] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2376.365282] RIP: 0033:0x7f33957fc10b [ 2376.367284] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2376.377974] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2376.382421] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2376.386861] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2376.391228] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2376.396931] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2376.401516] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2380.058048] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2380.062854] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2380.105798] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2380.113046] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2380.118023] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2380.126147] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2380.135001] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2380.149265] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2380.152848] RAX: 000000000000058a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2380.155559] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2380.158851] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2380.161899] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2380.164820] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2380.168103] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2380.171146] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2380.173809] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2380.179726] Call Trace: [ 2380.180964] [ 2380.183005] ? show_regs.cold.9+0x22/0x2f [ 2380.185713] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2380.188240] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2380.190823] ? watchdog+0x40/0x40 [ 2380.192728] ? hrtimer_interrupt+0x128/0x2c0 [ 2380.194746] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2380.197228] ? apic_timer_interrupt+0xf/0x20 [ 2380.199485] [ 2380.203053] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2380.206727] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2380.211890] do_raw_spin_lock+0x177/0x190 [ 2380.213596] _raw_spin_lock+0x21/0x30 [ 2380.215088] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2380.218042] process_one_work+0x2c8/0x700 [ 2380.219779] worker_thread+0x296/0x6e0 [ 2380.221567] ? rescuer_thread+0x570/0x570 [ 2380.223124] kthread+0x1d1/0x200 [ 2380.224760] ? set_kthread_struct+0x70/0x70 [ 2380.226618] ret_from_fork+0x1f/0x30 [ 2392.053011] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2392.056618] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2392.091661] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2392.097200] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2392.105922] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2392.108307] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2392.115175] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2392.131941] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2392.137256] RDX: 0000000000001895 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2392.142390] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2392.145595] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2392.156086] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2392.164507] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2392.168277] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2392.172618] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2392.175608] Call Trace: [ 2392.176675] [ 2392.177316] ? show_regs.cold.9+0x22/0x2f [ 2392.179306] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2392.181351] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2392.183562] ? watchdog+0x40/0x40 [ 2392.184971] ? hrtimer_interrupt+0x128/0x2c0 [ 2392.186870] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2392.189293] ? apic_timer_interrupt+0xf/0x20 [ 2392.191039] [ 2392.191984] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2392.194500] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2392.200210] do_raw_spin_lock+0x177/0x190 [ 2392.201652] _raw_spin_lock+0x21/0x30 [ 2392.203286] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2392.205630] obd_mod_free+0x46/0x140 [ptlrpc] [ 2392.207677] mdc_commit_open+0x91/0xb0 [mdc] [ 2392.209311] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2392.212508] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2392.214622] after_reply+0xe2c/0x1780 [ptlrpc] [ 2392.216565] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2392.218850] ? __next_timer_interrupt+0x160/0x160 [ 2392.220781] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2392.222427] ? do_wait_intr+0xf0/0xf0 [ 2392.223963] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2392.226359] kthread+0x1d1/0x200 [ 2392.227658] ? set_kthread_struct+0x70/0x70 [ 2392.229383] ret_from_fork+0x1f/0x30 [ 2404.061985] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2404.067741] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2404.127735] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2404.142211] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2404.155227] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2404.170619] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2404.199695] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2404.207242] RAX: 0000000000007a97 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2404.219569] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2404.227883] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2404.243696] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2404.259900] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2404.273376] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2404.289650] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2404.300925] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2404.314342] Call Trace: [ 2404.320691] [ 2404.321370] ? show_regs.cold.9+0x22/0x2f [ 2404.328233] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2404.336904] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2404.344325] ? watchdog+0x40/0x40 [ 2404.350214] ? hrtimer_interrupt+0x128/0x2c0 [ 2404.355674] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2404.362544] ? apic_timer_interrupt+0xf/0x20 [ 2404.369735] [ 2404.371891] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2404.379814] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2404.382427] do_raw_spin_lock+0x177/0x190 [ 2404.392332] _raw_spin_lock+0x21/0x30 [ 2404.395928] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2404.401877] ? do_raw_spin_unlock+0x75/0x190 [ 2404.405781] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2404.409987] ? do_wait_intr+0xf0/0xf0 [ 2404.411864] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2404.416616] mdc_reint+0x2d/0x110 [mdc] [ 2404.421670] mdc_unlink+0x658/0x870 [mdc] [ 2404.426236] lmv_unlink+0x34d/0xbd0 [lmv] [ 2404.432168] ll_unlink+0x24d/0x7f0 [lustre] [ 2404.437403] vfs_unlink+0x202/0x330 [ 2404.440797] do_unlinkat+0x2fa/0x460 [ 2404.443813] __x64_sys_unlink+0x37/0x50 [ 2404.446801] do_syscall_64+0xc1/0x440 [ 2404.448506] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2404.450746] RIP: 0033:0x7f33957fc10b [ 2404.451861] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2404.465571] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2404.471251] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2404.475837] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2404.481682] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2404.488581] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2404.495082] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2408.065272] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2408.071859] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2408.113301] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2408.126799] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2408.141462] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2408.149742] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2408.156650] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2408.172100] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2408.178193] RAX: 0000000000007f08 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2408.187155] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2408.192352] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2408.202111] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2408.209494] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2408.216412] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2408.224774] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2408.231199] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2408.240538] Call Trace: [ 2408.243437] [ 2408.246032] ? show_regs.cold.9+0x22/0x2f [ 2408.249172] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2408.255820] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2408.258749] ? watchdog+0x40/0x40 [ 2408.260843] ? hrtimer_interrupt+0x128/0x2c0 [ 2408.263026] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2408.266414] ? apic_timer_interrupt+0xf/0x20 [ 2408.269202] [ 2408.270104] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2408.273249] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2408.276241] do_raw_spin_lock+0x177/0x190 [ 2408.280195] _raw_spin_lock+0x21/0x30 [ 2408.283192] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2408.295230] process_one_work+0x2c8/0x700 [ 2408.299626] worker_thread+0x296/0x6e0 [ 2408.302260] ? rescuer_thread+0x570/0x570 [ 2408.309294] kthread+0x1d1/0x200 [ 2408.313773] ? set_kthread_struct+0x70/0x70 [ 2408.318196] ret_from_fork+0x1f/0x30 [ 2420.053053] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2420.057988] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2420.085641] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2420.099905] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2420.109846] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2420.113976] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2420.130357] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2420.135952] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2420.140362] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2420.144217] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2420.148463] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2420.152387] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2420.156642] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2420.167660] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2420.177221] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2420.184601] Call Trace: [ 2420.185517] [ 2420.186234] ? show_regs.cold.9+0x22/0x2f [ 2420.189231] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2420.192975] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2420.199619] ? watchdog+0x40/0x40 [ 2420.201557] ? hrtimer_interrupt+0x128/0x2c0 [ 2420.205061] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2420.208467] ? apic_timer_interrupt+0xf/0x20 [ 2420.211001] [ 2420.212429] ? native_safe_halt+0x2e/0x50 [ 2420.214360] kvm_wait+0xcd/0xe0 [ 2420.215983] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2420.224786] do_raw_spin_lock+0x177/0x190 [ 2420.229215] _raw_spin_lock+0x21/0x30 [ 2420.231406] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2420.234642] obd_mod_free+0x46/0x140 [ptlrpc] [ 2420.238124] mdc_commit_open+0x91/0xb0 [mdc] [ 2420.241799] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2420.246035] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2420.249717] after_reply+0xe2c/0x1780 [ptlrpc] [ 2420.252743] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2420.265771] ? __next_timer_interrupt+0x160/0x160 [ 2420.275136] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2420.284742] ? do_wait_intr+0xf0/0xf0 [ 2420.294223] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2420.305412] kthread+0x1d1/0x200 [ 2420.313128] ? set_kthread_struct+0x70/0x70 [ 2420.317360] ret_from_fork+0x1f/0x30 [ 2432.060063] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2432.063102] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2432.084090] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2432.090109] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2432.094115] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2432.096785] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2432.103236] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2432.106601] RAX: 0000000000007997 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2432.108808] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2432.114869] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2432.119032] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2432.123020] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2432.129955] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2432.135049] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2432.137683] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2432.140609] Call Trace: [ 2432.141827] [ 2432.142938] ? show_regs.cold.9+0x22/0x2f [ 2432.144912] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2432.146962] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2432.149257] ? watchdog+0x40/0x40 [ 2432.150746] ? hrtimer_interrupt+0x128/0x2c0 [ 2432.152575] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2432.154833] ? apic_timer_interrupt+0xf/0x20 [ 2432.156426] [ 2432.157540] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2432.159714] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2432.161771] do_raw_spin_lock+0x177/0x190 [ 2432.163451] _raw_spin_lock+0x21/0x30 [ 2432.164788] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2432.166726] ? do_raw_spin_unlock+0x75/0x190 [ 2432.168636] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2432.170668] ? do_wait_intr+0xf0/0xf0 [ 2432.171671] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2432.173816] mdc_reint+0x2d/0x110 [mdc] [ 2432.175539] mdc_unlink+0x658/0x870 [mdc] [ 2432.177178] lmv_unlink+0x34d/0xbd0 [lmv] [ 2432.179992] ll_unlink+0x24d/0x7f0 [lustre] [ 2432.182000] vfs_unlink+0x202/0x330 [ 2432.183686] do_unlinkat+0x2fa/0x460 [ 2432.185404] __x64_sys_unlink+0x37/0x50 [ 2432.186986] do_syscall_64+0xc1/0x440 [ 2432.189287] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2432.191354] RIP: 0033:0x7f33957fc10b [ 2432.192877] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2432.201518] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2432.205184] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2432.209196] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2432.213589] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2432.216904] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2432.219891] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2436.058032] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2436.061836] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2436.092559] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2436.098627] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2436.104058] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2436.110651] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2436.113328] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2436.134492] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 2436.141377] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2436.152514] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2436.160701] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2436.167606] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2436.171539] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2436.180817] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2436.187513] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2436.194205] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2436.199057] Call Trace: [ 2436.201150] [ 2436.204107] ? show_regs.cold.9+0x22/0x2f [ 2436.209465] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2436.212014] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2436.216406] ? watchdog+0x40/0x40 [ 2436.219029] ? hrtimer_interrupt+0x128/0x2c0 [ 2436.223045] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2436.226918] ? apic_timer_interrupt+0xf/0x20 [ 2436.232908] [ 2436.233954] ? native_safe_halt+0x2e/0x50 [ 2436.236715] kvm_wait+0xcd/0xe0 [ 2436.239339] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2436.242113] do_raw_spin_lock+0x177/0x190 [ 2436.245714] _raw_spin_lock+0x21/0x30 [ 2436.250394] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2436.251877] process_one_work+0x2c8/0x700 [ 2436.256443] worker_thread+0x296/0x6e0 [ 2436.258802] ? rescuer_thread+0x570/0x570 [ 2436.261464] kthread+0x1d1/0x200 [ 2436.263468] ? set_kthread_struct+0x70/0x70 [ 2436.266348] ret_from_fork+0x1f/0x30 [ 2448.056365] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2448.099155] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2448.142974] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2448.149772] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2448.153569] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2448.157055] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2448.166687] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2448.169384] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2448.172242] RDX: 0000000000007e1d RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2448.174836] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2448.177253] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2448.179691] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2448.182094] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2448.185299] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2448.187583] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2448.190084] Call Trace: [ 2448.191061] [ 2448.191854] ? show_regs.cold.9+0x22/0x2f [ 2448.193456] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2448.195211] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2448.196780] ? watchdog+0x40/0x40 [ 2448.198048] ? hrtimer_interrupt+0x128/0x2c0 [ 2448.199757] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2448.201577] ? apic_timer_interrupt+0xf/0x20 [ 2448.203238] [ 2448.203952] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2448.206033] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2448.208231] do_raw_spin_lock+0x177/0x190 [ 2448.209746] _raw_spin_lock+0x21/0x30 [ 2448.211061] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2448.213254] obd_mod_free+0x46/0x140 [ptlrpc] [ 2448.215206] mdc_commit_open+0x91/0xb0 [mdc] [ 2448.216810] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2448.218704] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2448.221030] after_reply+0xe2c/0x1780 [ptlrpc] [ 2448.223026] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2448.224648] ? __next_timer_interrupt+0x160/0x160 [ 2448.226078] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2448.227902] ? do_wait_intr+0xf0/0xf0 [ 2448.229315] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2448.231451] kthread+0x1d1/0x200 [ 2448.232748] ? set_kthread_struct+0x70/0x70 [ 2448.234384] ret_from_fork+0x1f/0x30 [ 2458.592070] INFO: task khugepaged:48 blocked for more than 120 seconds. [ 2458.605457] Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2458.621651] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2458.636947] task:khugepaged state:D stack:0 pid:48 ppid:2 flags:0x80004000 [ 2458.648771] Call Trace: [ 2458.657077] __schedule+0x351/0xcb0 [ 2458.662703] ? wait_for_completion+0xae/0x1e0 [ 2458.677193] schedule+0xc0/0x180 [ 2458.682403] schedule_timeout+0x126/0x190 [ 2458.687245] ? try_to_wake_up+0x80/0x710 [ 2458.696923] ? __prepare_to_swait+0x5b/0x90 [ 2458.712330] ? do_raw_spin_unlock+0x75/0x190 [ 2458.713721] wait_for_completion+0xf0/0x1e0 [ 2458.725721] flush_work+0x1f3/0x390 [ 2458.733671] ? flush_workqueue_prep_pwqs+0x200/0x200 [ 2458.735361] ? do_raw_spin_unlock+0x75/0x190 [ 2458.754030] ? __pagevec_release+0x50/0x50 [ 2458.768756] lru_add_drain_all+0x1d4/0x280 [ 2458.771346] khugepaged+0xa2/0x16f0 [ 2458.781680] ? woken_wake_function+0x30/0x30 [ 2458.790471] ? collapse_pte_mapped_thp+0x8a0/0x8a0 [ 2458.800500] kthread+0x1d1/0x200 [ 2458.801519] ? set_kthread_struct+0x70/0x70 [ 2458.807385] ret_from_fork+0x1f/0x30 [ 2460.061620] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2460.070237] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2460.161004] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2460.178872] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2460.187384] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2460.194697] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2460.227773] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 2460.249760] RAX: 0000000000005c6b RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2460.271821] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2460.281191] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2460.290837] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2460.304822] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2460.323007] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2460.337762] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2460.344799] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2460.359230] Call Trace: [ 2460.364413] [ 2460.369380] ? show_regs.cold.9+0x22/0x2f [ 2460.375656] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2460.386336] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2460.400727] ? watchdog+0x40/0x40 [ 2460.407407] ? hrtimer_interrupt+0x128/0x2c0 [ 2460.417838] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2460.424309] ? apic_timer_interrupt+0xf/0x20 [ 2460.425656] [ 2460.435599] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2460.446727] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2460.454132] do_raw_spin_lock+0x177/0x190 [ 2460.461549] _raw_spin_lock+0x21/0x30 [ 2460.476055] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2460.488375] ? do_raw_spin_unlock+0x75/0x190 [ 2460.498359] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2460.508956] ? do_wait_intr+0xf0/0xf0 [ 2460.515356] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2460.519614] mdc_reint+0x2d/0x110 [mdc] [ 2460.530481] mdc_unlink+0x658/0x870 [mdc] [ 2460.537588] lmv_unlink+0x34d/0xbd0 [lmv] [ 2460.540536] ll_unlink+0x24d/0x7f0 [lustre] [ 2460.546693] vfs_unlink+0x202/0x330 [ 2460.554482] do_unlinkat+0x2fa/0x460 [ 2460.560570] __x64_sys_unlink+0x37/0x50 [ 2460.565347] do_syscall_64+0xc1/0x440 [ 2460.570047] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2460.580413] RIP: 0033:0x7f33957fc10b [ 2460.582506] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2460.609098] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2460.617312] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2460.625740] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2460.638790] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2460.649709] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2460.657673] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2464.059067] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2464.068913] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2464.139425] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2464.151310] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2464.165786] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2464.168088] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2464.171118] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2464.199611] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2464.210375] RAX: 000000000000546d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2464.221808] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2464.233488] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2464.242530] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2464.250828] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2464.256474] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2464.263748] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2464.274443] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2464.281148] Call Trace: [ 2464.287526] [ 2464.288402] ? show_regs.cold.9+0x22/0x2f [ 2464.296737] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2464.307322] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2464.309048] ? watchdog+0x40/0x40 [ 2464.314358] ? hrtimer_interrupt+0x128/0x2c0 [ 2464.320218] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2464.326190] ? apic_timer_interrupt+0xf/0x20 [ 2464.329993] [ 2464.334037] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2464.339401] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2464.347164] do_raw_spin_lock+0x177/0x190 [ 2464.351647] _raw_spin_lock+0x21/0x30 [ 2464.355032] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2464.361181] process_one_work+0x2c8/0x700 [ 2464.365899] worker_thread+0x296/0x6e0 [ 2464.369732] ? rescuer_thread+0x570/0x570 [ 2464.373675] kthread+0x1d1/0x200 [ 2464.375376] ? set_kthread_struct+0x70/0x70 [ 2464.378215] ret_from_fork+0x1f/0x30 [ 2476.053721] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2476.074118] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2476.127075] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2476.133001] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2476.139048] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2476.144014] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2476.166767] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2476.174946] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2476.187830] RDX: 0000000000000731 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2476.197375] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2476.202624] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2476.206801] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2476.212329] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2476.217396] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2476.221208] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2476.224678] Call Trace: [ 2476.225714] [ 2476.227130] ? show_regs.cold.9+0x22/0x2f [ 2476.229744] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2476.232818] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2476.238897] ? watchdog+0x40/0x40 [ 2476.240408] ? hrtimer_interrupt+0x128/0x2c0 [ 2476.242815] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2476.246414] ? apic_timer_interrupt+0xf/0x20 [ 2476.248774] [ 2476.251101] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2476.256742] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2476.259630] do_raw_spin_lock+0x177/0x190 [ 2476.263199] _raw_spin_lock+0x21/0x30 [ 2476.269758] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2476.277413] obd_mod_free+0x46/0x140 [ptlrpc] [ 2476.287231] mdc_commit_open+0x91/0xb0 [mdc] [ 2476.292069] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2476.296741] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2476.302140] after_reply+0xe2c/0x1780 [ptlrpc] [ 2476.305341] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2476.309614] ? __next_timer_interrupt+0x160/0x160 [ 2476.315477] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2476.326207] ? do_wait_intr+0xf0/0xf0 [ 2476.331359] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2476.338249] kthread+0x1d1/0x200 [ 2476.341192] ? set_kthread_struct+0x70/0x70 [ 2476.347244] ret_from_fork+0x1f/0x30 [ 2488.059021] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2488.062103] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2488.081360] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2488.086323] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2488.089788] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2488.091678] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2488.099647] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2488.102811] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2488.105752] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2488.108538] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2488.111777] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2488.114505] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2488.117454] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2488.120592] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2488.122960] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2488.125603] Call Trace: [ 2488.126579] [ 2488.127369] ? show_regs.cold.9+0x22/0x2f [ 2488.129154] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2488.131219] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2488.133197] ? watchdog+0x40/0x40 [ 2488.134477] ? hrtimer_interrupt+0x128/0x2c0 [ 2488.136633] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2488.138431] ? apic_timer_interrupt+0xf/0x20 [ 2488.142675] [ 2488.143628] ? native_safe_halt+0x2e/0x50 [ 2488.145552] kvm_wait+0xcd/0xe0 [ 2488.148701] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2488.153079] do_raw_spin_lock+0x177/0x190 [ 2488.155118] _raw_spin_lock+0x21/0x30 [ 2488.157152] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2488.160107] ? do_raw_spin_unlock+0x75/0x190 [ 2488.161638] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2488.163980] ? do_wait_intr+0xf0/0xf0 [ 2488.165241] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2488.167149] mdc_reint+0x2d/0x110 [mdc] [ 2488.168457] mdc_unlink+0x658/0x870 [mdc] [ 2488.169446] lmv_unlink+0x34d/0xbd0 [lmv] [ 2488.171391] ll_unlink+0x24d/0x7f0 [lustre] [ 2488.173415] vfs_unlink+0x202/0x330 [ 2488.174791] do_unlinkat+0x2fa/0x460 [ 2488.176105] __x64_sys_unlink+0x37/0x50 [ 2488.177984] do_syscall_64+0xc1/0x440 [ 2488.179387] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2488.181574] RIP: 0033:0x7f33957fc10b [ 2488.182986] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2488.200271] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2488.205709] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2488.210841] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2488.216903] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2488.221022] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2488.224029] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2492.058024] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2492.062858] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2492.082577] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2492.087087] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2492.090573] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2492.093453] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2492.095636] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2492.104209] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2492.109563] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2492.112527] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2492.116376] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2492.120010] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2492.123136] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2492.125995] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2492.129557] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2492.131954] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2492.135643] Call Trace: [ 2492.136937] [ 2492.138000] ? show_regs.cold.9+0x22/0x2f [ 2492.139754] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2492.142177] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2492.144039] ? watchdog+0x40/0x40 [ 2492.145363] ? hrtimer_interrupt+0x128/0x2c0 [ 2492.147456] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2492.149182] ? apic_timer_interrupt+0xf/0x20 [ 2492.151110] [ 2492.152123] ? native_safe_halt+0x2e/0x50 [ 2492.154084] kvm_wait+0xcd/0xe0 [ 2492.155334] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2492.157562] do_raw_spin_lock+0x177/0x190 [ 2492.159449] _raw_spin_lock+0x21/0x30 [ 2492.161623] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2492.163972] process_one_work+0x2c8/0x700 [ 2492.166179] worker_thread+0x296/0x6e0 [ 2492.167696] ? rescuer_thread+0x570/0x570 [ 2492.169843] kthread+0x1d1/0x200 [ 2492.170932] ? set_kthread_struct+0x70/0x70 [ 2492.172893] ret_from_fork+0x1f/0x30 [ 2504.053076] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2504.057819] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2504.081123] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2504.088518] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2504.093756] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2504.096570] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2504.106731] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2504.111366] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2504.116228] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2504.120189] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2504.124224] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2504.128368] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2504.131829] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2504.136198] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2504.139648] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2504.143212] Call Trace: [ 2504.144699] [ 2504.145372] ? show_regs.cold.9+0x22/0x2f [ 2504.147459] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2504.150471] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2504.152938] ? watchdog+0x40/0x40 [ 2504.154895] ? hrtimer_interrupt+0x128/0x2c0 [ 2504.156966] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2504.159553] ? apic_timer_interrupt+0xf/0x20 [ 2504.161760] [ 2504.162785] ? native_safe_halt+0x2e/0x50 [ 2504.164771] kvm_wait+0xcd/0xe0 [ 2504.166198] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2504.168819] do_raw_spin_lock+0x177/0x190 [ 2504.171237] _raw_spin_lock+0x21/0x30 [ 2504.173165] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2504.175853] obd_mod_free+0x46/0x140 [ptlrpc] [ 2504.178512] mdc_commit_open+0x91/0xb0 [mdc] [ 2504.181279] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2504.184754] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2504.187858] after_reply+0xe2c/0x1780 [ptlrpc] [ 2504.190452] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2504.193575] ? __next_timer_interrupt+0x160/0x160 [ 2504.195913] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2504.198691] ? do_wait_intr+0xf0/0xf0 [ 2504.200270] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2504.203492] kthread+0x1d1/0x200 [ 2504.205215] ? set_kthread_struct+0x70/0x70 [ 2504.209018] ret_from_fork+0x1f/0x30 [ 2516.059133] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2516.071610] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2516.099570] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2516.104117] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2516.107578] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2516.114527] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2516.124630] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2516.130429] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2516.133827] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2516.139334] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2516.142367] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2516.151597] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2516.157064] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2516.160285] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2516.162450] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2516.165101] Call Trace: [ 2516.166214] [ 2516.167153] ? show_regs.cold.9+0x22/0x2f [ 2516.168688] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2516.170451] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2516.172100] ? watchdog+0x40/0x40 [ 2516.173288] ? hrtimer_interrupt+0x128/0x2c0 [ 2516.174645] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2516.176597] ? apic_timer_interrupt+0xf/0x20 [ 2516.178247] [ 2516.179129] ? native_safe_halt+0x2e/0x50 [ 2516.180727] kvm_wait+0xcd/0xe0 [ 2516.182210] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2516.184256] do_raw_spin_lock+0x177/0x190 [ 2516.185930] _raw_spin_lock+0x21/0x30 [ 2516.187414] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2516.189662] ? do_raw_spin_unlock+0x75/0x190 [ 2516.191121] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2516.193247] ? do_wait_intr+0xf0/0xf0 [ 2516.194552] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2516.196661] mdc_reint+0x2d/0x110 [mdc] [ 2516.198408] mdc_unlink+0x658/0x870 [mdc] [ 2516.199967] lmv_unlink+0x34d/0xbd0 [lmv] [ 2516.201279] ll_unlink+0x24d/0x7f0 [lustre] [ 2516.202957] vfs_unlink+0x202/0x330 [ 2516.204199] do_unlinkat+0x2fa/0x460 [ 2516.205440] __x64_sys_unlink+0x37/0x50 [ 2516.206892] do_syscall_64+0xc1/0x440 [ 2516.208238] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2516.210090] RIP: 0033:0x7f33957fc10b [ 2516.211160] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2516.217809] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2516.221017] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2516.224299] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2516.228324] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2516.230877] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2516.233642] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2518.027248] rcu: INFO: rcu_sched self-detected stall on CPU [ 2518.029653] rcu: 0-....: (241343 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=76646 [ 2518.039866] (t=420007 jiffies g=297885 q=4355) [ 2518.042575] NMI backtrace for cpu 0 [ 2518.048527] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2518.053941] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2518.061429] Call Trace: [ 2518.063718] [ 2518.064744] ? dump_stack+0xbb/0x10e [ 2518.068695] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 2518.074634] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 2518.078833] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 2518.083582] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 2518.086167] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 2518.090693] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 2518.093930] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 2518.097872] ? update_process_times+0x6d/0xc0 [ 2518.099697] ? tick_sched_handle.isra.18+0x27/0x90 [ 2518.101843] ? tick_sched_timer+0x53/0xb0 [ 2518.108106] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2518.112773] ? tick_sched_handle.isra.18+0x90/0x90 [ 2518.115986] ? hrtimer_interrupt+0x128/0x2c0 [ 2518.120286] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2518.122888] ? apic_timer_interrupt+0xf/0x20 [ 2518.125186] [ 2518.126354] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2518.133891] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2518.136959] ? do_raw_spin_lock+0x177/0x190 [ 2518.139429] ? _raw_spin_lock+0x21/0x30 [ 2518.142663] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2518.145599] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2518.153346] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2518.156961] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2518.164278] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2518.173896] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2518.177648] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2518.184756] ? __next_timer_interrupt+0x160/0x160 [ 2518.187266] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2518.196261] ? do_wait_intr+0xf0/0xf0 [ 2518.199827] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2518.205887] ? kthread+0x1d1/0x200 [ 2518.211580] ? set_kthread_struct+0x70/0x70 [ 2518.213352] ? ret_from_fork+0x1f/0x30 [ 2520.058873] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2520.065500] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2520.144651] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2520.159352] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2520.172108] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2520.177428] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2520.188221] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2520.221073] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2520.232875] RAX: 0000000000007391 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2520.236992] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2520.248290] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2520.260316] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2520.264334] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2520.269321] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2520.274648] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2520.278755] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2520.282801] Call Trace: [ 2520.284444] [ 2520.285190] ? show_regs.cold.9+0x22/0x2f [ 2520.287810] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2520.298264] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2520.303916] ? watchdog+0x40/0x40 [ 2520.305081] ? hrtimer_interrupt+0x128/0x2c0 [ 2520.314361] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2520.323274] ? apic_timer_interrupt+0xf/0x20 [ 2520.329596] [ 2520.331829] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2520.339695] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2520.348167] do_raw_spin_lock+0x177/0x190 [ 2520.349985] _raw_spin_lock+0x21/0x30 [ 2520.351300] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2520.361350] process_one_work+0x2c8/0x700 [ 2520.365445] worker_thread+0x296/0x6e0 [ 2520.372229] ? rescuer_thread+0x570/0x570 [ 2520.374848] kthread+0x1d1/0x200 [ 2520.377403] ? set_kthread_struct+0x70/0x70 [ 2520.381112] ret_from_fork+0x1f/0x30 [ 2544.054105] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2544.060900] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2544.063082] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) [ 2544.070905] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2544.070995] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2544.071004] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2544.071007] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2544.071283] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2544.071288] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2544.071292] RAX: 00000000000057ab RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2544.071295] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2544.071297] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2544.071299] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2544.071302] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2544.071305] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2544.071308] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2544.071369] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2544.071377] Call Trace: [ 2544.071385] [ 2544.071393] ? show_regs.cold.9+0x22/0x2f [ 2544.071403] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2544.071410] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2544.071414] ? watchdog+0x40/0x40 [ 2544.071419] ? hrtimer_interrupt+0x128/0x2c0 [ 2544.071424] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2544.071433] ? apic_timer_interrupt+0xf/0x20 [ 2544.071438] [ 2544.071441] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2544.071446] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2544.071450] do_raw_spin_lock+0x177/0x190 [ 2544.071456] _raw_spin_lock+0x21/0x30 [ 2544.071462] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2544.072139] ? do_raw_spin_unlock+0x75/0x190 [ 2544.072148] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2544.072811] ? do_wait_intr+0xf0/0xf0 [ 2544.072818] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2544.073014] mdc_reint+0x2d/0x110 [mdc] [ 2544.074161] mdc_unlink+0x658/0x870 [mdc] [ 2544.088721] mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2544.088790] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2544.088798] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2544.088801] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2544.088819] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2544.088823] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2544.088828] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2544.088830] RDX: 0000000000007c8c RSI: 0000000000000008 RDI: ffffffff91001000 [ 2544.088832] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2544.088835] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2544.088838] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2544.088841] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2544.088843] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2544.088845] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2544.088854] Call Trace: [ 2544.088861] [ 2544.088868] ? show_regs.cold.9+0x22/0x2f [ 2544.088884] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2544.088892] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2544.088896] ? watchdog+0x40/0x40 [ 2544.088901] ? hrtimer_interrupt+0x128/0x2c0 [ 2544.088921] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2544.088935] ? apic_timer_interrupt+0xf/0x20 [ 2544.088940] [ 2544.088943] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2544.088948] ? __pv_queued_spin_lock_slowpath+0x650/0x6d0 [ 2544.088952] do_raw_spin_lock+0x177/0x190 [ 2544.088957] _raw_spin_lock+0x21/0x30 [ 2544.088962] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2544.121500] lmv_unlink+0x34d/0xbd0 [lmv] [ 2544.128499] obd_mod_free+0x46/0x140 [ptlrpc] [ 2544.134708] ll_unlink+0x24d/0x7f0 [lustre] [ 2544.135139] vfs_unlink+0x202/0x330 [ 2544.135153] do_unlinkat+0x2fa/0x460 [ 2544.135159] __x64_sys_unlink+0x37/0x50 [ 2544.135163] do_syscall_64+0xc1/0x440 [ 2544.135170] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2544.135180] RIP: 0033:0x7f33957fc10b [ 2544.135261] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2544.135266] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2544.135273] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2544.135276] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2544.135279] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2544.135281] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2544.135284] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2544.547386] mdc_commit_open+0x91/0xb0 [mdc] [ 2544.551557] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2544.556376] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2544.559466] after_reply+0xe2c/0x1780 [ptlrpc] [ 2544.562462] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2544.570389] ? __next_timer_interrupt+0x160/0x160 [ 2544.575199] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2544.578668] ? do_wait_intr+0xf0/0xf0 [ 2544.580683] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2544.584185] kthread+0x1d1/0x200 [ 2544.587104] ? set_kthread_struct+0x70/0x70 [ 2544.594473] ret_from_fork+0x1f/0x30 [ 2548.064612] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2548.075943] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2548.133605] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2548.145162] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2548.153570] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2548.162574] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2548.168512] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2548.190726] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2548.196368] RAX: 0000000000006cc6 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2548.201660] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2548.209654] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2548.213040] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2548.220167] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2548.226966] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2548.233213] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2548.240171] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2548.249603] Call Trace: [ 2548.256351] [ 2548.259595] ? show_regs.cold.9+0x22/0x2f [ 2548.262926] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2548.272184] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2548.275132] ? watchdog+0x40/0x40 [ 2548.277998] ? hrtimer_interrupt+0x128/0x2c0 [ 2548.283363] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2548.287429] ? apic_timer_interrupt+0xf/0x20 [ 2548.294596] [ 2548.299213] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2548.306701] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2548.310713] do_raw_spin_lock+0x177/0x190 [ 2548.319101] _raw_spin_lock+0x21/0x30 [ 2548.327648] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2548.337731] process_one_work+0x2c8/0x700 [ 2548.342728] worker_thread+0x296/0x6e0 [ 2548.348973] ? rescuer_thread+0x570/0x570 [ 2548.350612] kthread+0x1d1/0x200 [ 2548.355425] ? set_kthread_struct+0x70/0x70 [ 2548.360628] ret_from_fork+0x1f/0x30 [ 2572.053060] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2572.059076] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2572.060059] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2572.060405] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2572.060412] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2572.060415] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2572.060430] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2572.060434] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2572.060438] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2572.060440] RDX: 0000000000004bf7 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2572.060442] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2572.060444] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2572.060447] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2572.060454] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2572.060457] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2572.060459] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2572.060467] Call Trace: [ 2572.060478] [ 2572.060485] ? show_regs.cold.9+0x22/0x2f [ 2572.060501] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2572.060508] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2572.060512] ? watchdog+0x40/0x40 [ 2572.060516] ? hrtimer_interrupt+0x128/0x2c0 [ 2572.060520] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2572.060529] ? apic_timer_interrupt+0xf/0x20 [ 2572.060534] [ 2572.060536] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2572.060541] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2572.060546] do_raw_spin_lock+0x177/0x190 [ 2572.060550] _raw_spin_lock+0x21/0x30 [ 2572.060555] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2572.061128] obd_mod_free+0x46/0x140 [ptlrpc] [ 2572.061355] mdc_commit_open+0x91/0xb0 [mdc] [ 2572.061403] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2572.065982] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2572.066093] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2572.066101] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2572.066103] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2572.066116] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2572.066126] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2572.066132] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2572.066134] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2572.066136] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2572.066138] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2572.066140] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2572.066142] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2572.066149] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2572.066151] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2572.066158] Call Trace: [ 2572.066167] [ 2572.066173] ? show_regs.cold.9+0x22/0x2f [ 2572.066184] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2572.066191] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2572.066196] ? watchdog+0x40/0x40 [ 2572.066200] ? hrtimer_interrupt+0x128/0x2c0 [ 2572.066204] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2572.066212] ? apic_timer_interrupt+0xf/0x20 [ 2572.066216] [ 2572.066223] ? native_safe_halt+0x2e/0x50 [ 2572.066229] kvm_wait+0xcd/0xe0 [ 2572.102403] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2572.103157] after_reply+0xe2c/0x1780 [ptlrpc] [ 2572.110199] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2572.110224] do_raw_spin_lock+0x177/0x190 [ 2572.110230] _raw_spin_lock+0x21/0x30 [ 2572.110239] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2572.110538] ? do_raw_spin_unlock+0x75/0x190 [ 2572.110545] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2572.119631] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2572.125097] ? do_wait_intr+0xf0/0xf0 [ 2572.125118] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2572.125379] mdc_reint+0x2d/0x110 [mdc] [ 2572.125422] mdc_unlink+0x658/0x870 [mdc] [ 2572.125449] lmv_unlink+0x34d/0xbd0 [lmv] [ 2572.125485] ll_unlink+0x24d/0x7f0 [lustre] [ 2572.125598] vfs_unlink+0x202/0x330 [ 2572.125609] do_unlinkat+0x2fa/0x460 [ 2572.125614] __x64_sys_unlink+0x37/0x50 [ 2572.125618] do_syscall_64+0xc1/0x440 [ 2572.125626] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2572.125836] RIP: 0033:0x7f33957fc10b [ 2572.125843] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2572.125847] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2572.125853] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2572.125855] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2572.125856] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2572.125859] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2572.125861] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2572.449828] ? __next_timer_interrupt+0x160/0x160 [ 2572.451806] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2572.453880] ? do_wait_intr+0xf0/0xf0 [ 2572.454988] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2572.457131] kthread+0x1d1/0x200 [ 2572.458539] ? set_kthread_struct+0x70/0x70 [ 2572.460520] ret_from_fork+0x1f/0x30 [ 2576.058039] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2576.062256] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2576.114645] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2576.126310] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2576.141625] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2576.150459] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2576.155772] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2576.187334] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2576.207531] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2576.215292] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2576.230673] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2576.234990] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2576.245934] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2576.252104] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2576.259532] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2576.263611] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2576.271474] Call Trace: [ 2576.277367] [ 2576.279106] ? show_regs.cold.9+0x22/0x2f [ 2576.282896] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2576.288040] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2576.292061] ? watchdog+0x40/0x40 [ 2576.299315] ? hrtimer_interrupt+0x128/0x2c0 [ 2576.302771] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2576.307871] ? apic_timer_interrupt+0xf/0x20 [ 2576.313623] [ 2576.314905] ? native_safe_halt+0x2e/0x50 [ 2576.317835] kvm_wait+0xcd/0xe0 [ 2576.320956] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2576.323743] do_raw_spin_lock+0x177/0x190 [ 2576.328629] _raw_spin_lock+0x21/0x30 [ 2576.334015] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2576.338935] process_one_work+0x2c8/0x700 [ 2576.346017] worker_thread+0x296/0x6e0 [ 2576.351864] ? rescuer_thread+0x570/0x570 [ 2576.354896] kthread+0x1d1/0x200 [ 2576.357202] ? set_kthread_struct+0x70/0x70 [ 2576.359762] ret_from_fork+0x1f/0x30 [ 2600.053183] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2600.059033] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2600.068882] Modules linked in: [ 2600.072474] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2600.072568] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2600.072575] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2600.072579] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2600.072593] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2600.072598] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2600.072602] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2600.072604] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2600.072606] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2600.072609] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2600.072611] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2600.072613] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2600.072616] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2600.072618] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2600.072627] Call Trace: [ 2600.072632] [ 2600.072639] ? show_regs.cold.9+0x22/0x2f [ 2600.072649] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2600.072656] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2600.072661] ? watchdog+0x40/0x40 [ 2600.072672] ? hrtimer_interrupt+0x128/0x2c0 [ 2600.072676] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2600.072684] ? apic_timer_interrupt+0xf/0x20 [ 2600.077562] lustre(O) [ 2600.109075] [ 2600.109100] ? native_safe_halt+0x2e/0x50 [ 2600.109116] kvm_wait+0xcd/0xe0 [ 2600.109128] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2600.109139] do_raw_spin_lock+0x177/0x190 [ 2600.109143] _raw_spin_lock+0x21/0x30 [ 2600.109151] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2600.109564] ? do_raw_spin_unlock+0x75/0x190 [ 2600.109570] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2600.109856] ? do_wait_intr+0xf0/0xf0 [ 2600.109865] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2600.110092] mdc_reint+0x2d/0x110 [mdc] [ 2600.110140] mdc_unlink+0x658/0x870 [mdc] [ 2600.110295] lmv_unlink+0x34d/0xbd0 [lmv] [ 2600.110337] ll_unlink+0x24d/0x7f0 [lustre] [ 2600.110491] vfs_unlink+0x202/0x330 [ 2600.110501] do_unlinkat+0x2fa/0x460 [ 2600.110507] __x64_sys_unlink+0x37/0x50 [ 2600.110521] do_syscall_64+0xc1/0x440 [ 2600.110530] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2600.110537] RIP: 0033:0x7f33957fc10b [ 2600.110545] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2600.110550] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2600.110556] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2600.110558] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2600.110561] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2600.110563] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2600.110566] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2600.334365] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2600.376795] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2600.390900] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2600.398320] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2600.401943] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2600.418628] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2600.424051] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2600.427631] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2600.434815] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2600.438635] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2600.442485] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2600.453412] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2600.458880] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2600.463518] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2600.467062] Call Trace: [ 2600.468120] [ 2600.468811] ? show_regs.cold.9+0x22/0x2f [ 2600.470160] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2600.479953] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2600.483823] ? watchdog+0x40/0x40 [ 2600.484911] ? hrtimer_interrupt+0x128/0x2c0 [ 2600.487801] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2600.491074] ? apic_timer_interrupt+0xf/0x20 [ 2600.494679] [ 2600.495652] ? native_safe_halt+0x2e/0x50 [ 2600.502222] kvm_wait+0xcd/0xe0 [ 2600.506365] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2600.510751] do_raw_spin_lock+0x177/0x190 [ 2600.512924] _raw_spin_lock+0x21/0x30 [ 2600.515425] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2600.518524] obd_mod_free+0x46/0x140 [ptlrpc] [ 2600.521037] mdc_commit_open+0x91/0xb0 [mdc] [ 2600.524151] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2600.530148] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2600.533967] after_reply+0xe2c/0x1780 [ptlrpc] [ 2600.537583] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2600.543182] ? __next_timer_interrupt+0x160/0x160 [ 2600.545888] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2600.554611] ? do_wait_intr+0xf0/0xf0 [ 2600.556398] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2600.559392] kthread+0x1d1/0x200 [ 2600.562439] ? set_kthread_struct+0x70/0x70 [ 2600.569248] ret_from_fork+0x1f/0x30 [ 2604.059285] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2604.066904] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2604.127624] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2604.138252] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2604.147392] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2604.153907] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2604.160950] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2604.180782] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2604.187969] RAX: 000000000000597a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2604.194442] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2604.197833] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2604.204895] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2604.209115] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2604.213336] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2604.219844] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2604.223774] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2604.230681] Call Trace: [ 2604.232355] [ 2604.233603] ? show_regs.cold.9+0x22/0x2f [ 2604.235870] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2604.238086] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2604.246211] ? watchdog+0x40/0x40 [ 2604.249894] ? hrtimer_interrupt+0x128/0x2c0 [ 2604.256080] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2604.259488] ? apic_timer_interrupt+0xf/0x20 [ 2604.264160] [ 2604.265355] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2604.270840] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2604.275250] do_raw_spin_lock+0x177/0x190 [ 2604.277556] _raw_spin_lock+0x21/0x30 [ 2604.280213] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2604.282447] process_one_work+0x2c8/0x700 [ 2604.283816] worker_thread+0x296/0x6e0 [ 2604.285211] ? rescuer_thread+0x570/0x570 [ 2604.286879] kthread+0x1d1/0x200 [ 2604.288589] ? set_kthread_struct+0x70/0x70 [ 2604.290630] ret_from_fork+0x1f/0x30 [ 2628.057316] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2628.059121] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2628.073723] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2628.073807] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2628.073815] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2628.073818] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2628.073835] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2628.073839] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2628.073844] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2628.073846] RDX: 00000000000075b2 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2628.073848] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2628.073850] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2628.073852] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2628.073854] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2628.073857] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2628.073859] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2628.073869] Call Trace: [ 2628.073873] [ 2628.073879] ? show_regs.cold.9+0x22/0x2f [ 2628.073890] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2628.073896] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2628.073900] ? watchdog+0x40/0x40 [ 2628.078992] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2628.079381] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2628.079389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2628.079392] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2628.079406] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2628.079410] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2628.079414] RAX: 000000000000658a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2628.079417] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2628.079419] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2628.079421] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2628.079424] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2628.079433] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2628.079436] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2628.079442] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2628.079450] Call Trace: [ 2628.079455] [ 2628.079461] ? show_regs.cold.9+0x22/0x2f [ 2628.079476] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2628.079484] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2628.079489] ? watchdog+0x40/0x40 [ 2628.079498] ? hrtimer_interrupt+0x128/0x2c0 [ 2628.079506] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2628.079529] ? apic_timer_interrupt+0xf/0x20 [ 2628.079535] [ 2628.079538] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2628.079544] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2628.079548] do_raw_spin_lock+0x177/0x190 [ 2628.079553] _raw_spin_lock+0x21/0x30 [ 2628.079559] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2628.114269] ? hrtimer_interrupt+0x128/0x2c0 [ 2628.114292] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2628.114305] ? apic_timer_interrupt+0xf/0x20 [ 2628.114310] [ 2628.114316] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2628.114323] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2628.114327] do_raw_spin_lock+0x177/0x190 [ 2628.114333] _raw_spin_lock+0x21/0x30 [ 2628.114339] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2628.115346] obd_mod_free+0x46/0x140 [ptlrpc] [ 2628.123147] ? do_raw_spin_unlock+0x75/0x190 [ 2628.123174] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2628.128969] mdc_commit_open+0x91/0xb0 [mdc] [ 2628.129090] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2628.133325] ? do_wait_intr+0xf0/0xf0 [ 2628.133351] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2628.145705] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2628.146263] after_reply+0xe2c/0x1780 [ptlrpc] [ 2628.146515] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2628.146809] ? __next_timer_interrupt+0x160/0x160 [ 2628.146821] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2628.147158] ? do_wait_intr+0xf0/0xf0 [ 2628.147169] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2628.147412] kthread+0x1d1/0x200 [ 2628.147419] ? set_kthread_struct+0x70/0x70 [ 2628.147423] ret_from_fork+0x1f/0x30 [ 2628.441096] mdc_reint+0x2d/0x110 [mdc] [ 2628.443313] mdc_unlink+0x658/0x870 [mdc] [ 2628.444966] lmv_unlink+0x34d/0xbd0 [lmv] [ 2628.447161] ll_unlink+0x24d/0x7f0 [lustre] [ 2628.451110] vfs_unlink+0x202/0x330 [ 2628.453828] do_unlinkat+0x2fa/0x460 [ 2628.456763] __x64_sys_unlink+0x37/0x50 [ 2628.459283] do_syscall_64+0xc1/0x440 [ 2628.461350] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2628.465021] RIP: 0033:0x7f33957fc10b [ 2628.467494] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2628.481356] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2628.486171] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2628.490642] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2628.495636] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2628.499787] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2628.504753] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2632.061069] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2632.064692] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2632.123371] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2632.140249] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2632.158682] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2632.168978] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2632.177408] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2632.201535] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2632.209355] RAX: 0000000000006188 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2632.217045] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2632.225866] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2632.233053] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2632.236906] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2632.243692] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2632.252428] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2632.260298] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2632.270624] Call Trace: [ 2632.272763] [ 2632.279146] ? show_regs.cold.9+0x22/0x2f [ 2632.284635] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2632.290528] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2632.298652] ? watchdog+0x40/0x40 [ 2632.302864] ? hrtimer_interrupt+0x128/0x2c0 [ 2632.308888] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2632.315618] ? apic_timer_interrupt+0xf/0x20 [ 2632.324341] [ 2632.325694] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2632.336505] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2632.345725] do_raw_spin_lock+0x177/0x190 [ 2632.352099] _raw_spin_lock+0x21/0x30 [ 2632.358632] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2632.373695] process_one_work+0x2c8/0x700 [ 2632.376565] worker_thread+0x296/0x6e0 [ 2632.383388] ? rescuer_thread+0x570/0x570 [ 2632.388971] kthread+0x1d1/0x200 [ 2632.396908] ? set_kthread_struct+0x70/0x70 [ 2632.406057] ret_from_fork+0x1f/0x30 [ 2656.053830] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2656.059233] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2656.063629] Modules linked in: [ 2656.079137] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2656.079228] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2656.079235] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2656.079238] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2656.079263] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2656.079268] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2656.079273] RAX: 00000000000009fd RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2656.079275] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2656.079277] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2656.079279] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2656.079280] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2656.079284] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2656.079287] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2656.079289] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2656.079297] Call Trace: [ 2656.079303] [ 2656.079310] ? show_regs.cold.9+0x22/0x2f [ 2656.079323] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2656.079331] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2656.079335] ? watchdog+0x40/0x40 [ 2656.079340] ? hrtimer_interrupt+0x128/0x2c0 [ 2656.079343] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2656.079353] ? apic_timer_interrupt+0xf/0x20 [ 2656.079357] [ 2656.079360] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2656.079365] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2656.079370] do_raw_spin_lock+0x177/0x190 [ 2656.079375] _raw_spin_lock+0x21/0x30 [ 2656.079380] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2656.079727] ? do_raw_spin_unlock+0x75/0x190 [ 2656.079734] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2656.079939] ? do_wait_intr+0xf0/0xf0 [ 2656.079946] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2656.080148] mdc_reint+0x2d/0x110 [mdc] [ 2656.080195] mdc_unlink+0x658/0x870 [mdc] [ 2656.080227] lmv_unlink+0x34d/0xbd0 [lmv] [ 2656.080263] ll_unlink+0x24d/0x7f0 [lustre] [ 2656.080393] vfs_unlink+0x202/0x330 [ 2656.080415] do_unlinkat+0x2fa/0x460 [ 2656.080420] __x64_sys_unlink+0x37/0x50 [ 2656.080422] do_syscall_64+0xc1/0x440 [ 2656.080430] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2656.080438] RIP: 0033:0x7f33957fc10b [ 2656.080447] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2656.080450] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2656.080455] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2656.080458] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2656.080461] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2656.080462] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2656.080465] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2656.509735] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2656.543613] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2656.551697] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2656.560561] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2656.564366] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2656.577371] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2656.580578] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2656.584094] RDX: 0000000000006ea2 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2656.589718] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2656.593571] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2656.596929] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2656.600615] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2656.603750] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2656.606612] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2656.610528] Call Trace: [ 2656.612161] [ 2656.613414] ? show_regs.cold.9+0x22/0x2f [ 2656.615309] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2656.617776] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2656.620321] ? watchdog+0x40/0x40 [ 2656.621933] ? hrtimer_interrupt+0x128/0x2c0 [ 2656.624408] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2656.626617] ? apic_timer_interrupt+0xf/0x20 [ 2656.628764] [ 2656.629706] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2656.632206] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2656.635082] do_raw_spin_lock+0x177/0x190 [ 2656.636780] _raw_spin_lock+0x21/0x30 [ 2656.638599] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2656.640883] obd_mod_free+0x46/0x140 [ptlrpc] [ 2656.643420] mdc_commit_open+0x91/0xb0 [mdc] [ 2656.645774] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2656.649162] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2656.651613] after_reply+0xe2c/0x1780 [ptlrpc] [ 2656.653425] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2656.656208] ? __next_timer_interrupt+0x160/0x160 [ 2656.658899] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2656.661424] ? do_wait_intr+0xf0/0xf0 [ 2656.663310] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2656.666561] kthread+0x1d1/0x200 [ 2656.668193] ? set_kthread_struct+0x70/0x70 [ 2656.670152] ret_from_fork+0x1f/0x30 [ 2660.058033] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2660.063424] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2660.089306] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2660.093782] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2660.100623] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2660.104820] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2660.107235] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2660.115652] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2660.120175] RAX: 0000000000006f2c RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2660.125231] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2660.127676] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2660.133470] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2660.140719] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2660.146378] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2660.150640] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2660.155716] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2660.160345] Call Trace: [ 2660.162388] [ 2660.163373] ? show_regs.cold.9+0x22/0x2f [ 2660.165029] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2660.168315] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2660.170375] ? watchdog+0x40/0x40 [ 2660.174017] ? hrtimer_interrupt+0x128/0x2c0 [ 2660.175333] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2660.177699] ? apic_timer_interrupt+0xf/0x20 [ 2660.180121] [ 2660.181096] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2660.182900] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2660.184837] do_raw_spin_lock+0x177/0x190 [ 2660.186237] _raw_spin_lock+0x21/0x30 [ 2660.191418] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2660.193682] process_one_work+0x2c8/0x700 [ 2660.201010] worker_thread+0x296/0x6e0 [ 2660.202591] ? rescuer_thread+0x570/0x570 [ 2660.207803] kthread+0x1d1/0x200 [ 2660.208723] ? set_kthread_struct+0x70/0x70 [ 2660.209871] ret_from_fork+0x1f/0x30 [ 2684.053025] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2684.057611] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2684.059055] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2684.080925] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2684.085105] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) [ 2684.092189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2684.092192] mdc(O) lov(O) osc(O) lmv(O) [ 2684.100911] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2684.101008] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2684.101012] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 2684.101018] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2684.101021] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2684.101023] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2684.101025] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2684.101028] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2684.101031] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2684.101034] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2684.101036] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2684.101047] Call Trace: [ 2684.101068] [ 2684.105378] ec(O) [ 2684.107090] ? show_regs.cold.9+0x22/0x2f [ 2684.112292] fid(O) [ 2684.127863] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2684.127894] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2684.127899] ? watchdog+0x40/0x40 [ 2684.127903] ? hrtimer_interrupt+0x128/0x2c0 [ 2684.127908] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2684.127917] ? apic_timer_interrupt+0xf/0x20 [ 2684.127922] [ 2684.127928] ? native_safe_halt+0x2e/0x50 [ 2684.127933] kvm_wait+0xcd/0xe0 [ 2684.127943] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2684.127951] do_raw_spin_lock+0x177/0x190 [ 2684.127955] _raw_spin_lock+0x21/0x30 [ 2684.127961] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2684.131541] fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2684.131598] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2684.131606] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2684.131609] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2684.131627] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2684.131631] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2684.131636] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2684.131638] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2684.131654] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2684.131657] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2684.131659] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2684.131663] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2684.131667] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2684.131669] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2684.131677] Call Trace: [ 2684.131684] [ 2684.131690] ? show_regs.cold.9+0x22/0x2f [ 2684.131704] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2684.131711] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2684.131716] ? watchdog+0x40/0x40 [ 2684.131720] ? hrtimer_interrupt+0x128/0x2c0 [ 2684.131723] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2684.131731] ? apic_timer_interrupt+0xf/0x20 [ 2684.131736] [ 2684.131738] ? native_safe_halt+0x2e/0x50 [ 2684.131742] kvm_wait+0xcd/0xe0 [ 2684.131751] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2684.131758] do_raw_spin_lock+0x177/0x190 [ 2684.131763] _raw_spin_lock+0x21/0x30 [ 2684.131769] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2684.132698] ? do_raw_spin_unlock+0x75/0x190 [ 2684.132705] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2684.139237] obd_mod_free+0x46/0x140 [ptlrpc] [ 2684.142354] ? do_wait_intr+0xf0/0xf0 [ 2684.146221] mdc_commit_open+0x91/0xb0 [mdc] [ 2684.146285] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2684.157954] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2684.161284] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2684.161723] after_reply+0xe2c/0x1780 [ptlrpc] [ 2684.162074] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2684.162266] ? __next_timer_interrupt+0x160/0x160 [ 2684.162278] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2684.162492] ? do_wait_intr+0xf0/0xf0 [ 2684.162503] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2684.162714] kthread+0x1d1/0x200 [ 2684.162721] ? set_kthread_struct+0x70/0x70 [ 2684.162725] ret_from_fork+0x1f/0x30 [ 2684.520088] mdc_reint+0x2d/0x110 [mdc] [ 2684.521124] mdc_unlink+0x658/0x870 [mdc] [ 2684.522694] lmv_unlink+0x34d/0xbd0 [lmv] [ 2684.524150] ll_unlink+0x24d/0x7f0 [lustre] [ 2684.533450] vfs_unlink+0x202/0x330 [ 2684.536902] do_unlinkat+0x2fa/0x460 [ 2684.540052] __x64_sys_unlink+0x37/0x50 [ 2684.544062] do_syscall_64+0xc1/0x440 [ 2684.547089] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2684.553415] RIP: 0033:0x7f33957fc10b [ 2684.554653] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2684.581798] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2684.589282] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2684.593483] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2684.597459] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2684.605314] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2684.612566] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2688.058512] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2688.062601] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2688.089827] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2688.096748] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2688.102705] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2688.110339] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2688.114883] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2688.125689] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2688.131756] RAX: 000000000000671f RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2688.135499] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2688.139403] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2688.142617] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2688.145678] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2688.148630] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2688.152157] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2688.154814] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2688.158607] Call Trace: [ 2688.159682] [ 2688.160545] ? show_regs.cold.9+0x22/0x2f [ 2688.162406] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2688.164366] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2688.166201] ? watchdog+0x40/0x40 [ 2688.167489] ? hrtimer_interrupt+0x128/0x2c0 [ 2688.169237] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2688.171124] ? apic_timer_interrupt+0xf/0x20 [ 2688.173393] [ 2688.174248] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2688.176819] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2688.178866] do_raw_spin_lock+0x177/0x190 [ 2688.179920] _raw_spin_lock+0x21/0x30 [ 2688.180967] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2688.186100] process_one_work+0x2c8/0x700 [ 2688.189548] worker_thread+0x296/0x6e0 [ 2688.194664] ? rescuer_thread+0x570/0x570 [ 2688.199470] kthread+0x1d1/0x200 [ 2688.202138] ? set_kthread_struct+0x70/0x70 [ 2688.203753] ret_from_fork+0x1f/0x30 [ 2698.030121] rcu: INFO: rcu_sched self-detected stall on CPU [ 2698.035060] rcu: 0-....: (357419 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=110773 [ 2698.044882] (t=600010 jiffies g=297885 q=5089) [ 2698.048296] NMI backtrace for cpu 0 [ 2698.049784] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2698.055807] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2698.062291] Call Trace: [ 2698.063390] [ 2698.064575] ? dump_stack+0xbb/0x10e [ 2698.066094] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 2698.068617] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 2698.070744] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 2698.075604] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 2698.080044] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 2698.083495] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 2698.085904] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 2698.095209] ? update_process_times+0x6d/0xc0 [ 2698.097610] ? tick_sched_handle.isra.18+0x27/0x90 [ 2698.100343] ? tick_sched_timer+0x53/0xb0 [ 2698.106321] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2698.111575] ? tick_sched_handle.isra.18+0x90/0x90 [ 2698.115704] ? hrtimer_interrupt+0x128/0x2c0 [ 2698.119654] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2698.121887] ? apic_timer_interrupt+0xf/0x20 [ 2698.123947] [ 2698.124658] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2698.127978] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2698.131893] ? do_raw_spin_lock+0x177/0x190 [ 2698.135642] ? _raw_spin_lock+0x21/0x30 [ 2698.137777] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2698.142422] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2698.145825] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2698.150485] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2698.154549] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2698.160323] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2698.166157] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2698.170014] ? __next_timer_interrupt+0x160/0x160 [ 2698.172936] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2698.177770] ? do_wait_intr+0xf0/0xf0 [ 2698.181297] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2698.185327] ? kthread+0x1d1/0x200 [ 2698.188274] ? set_kthread_struct+0x70/0x70 [ 2698.191931] ? ret_from_fork+0x1f/0x30 [ 2712.061740] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2712.065118] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2712.126051] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2712.135142] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2712.144235] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2712.146581] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2712.161178] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2712.166001] RAX: 0000000000007e07 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2712.179292] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2712.185804] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2712.194037] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2712.208646] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2712.217624] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2712.227664] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2712.234356] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2712.240361] Call Trace: [ 2712.243743] [ 2712.245717] ? show_regs.cold.9+0x22/0x2f [ 2712.250401] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2712.252176] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2712.254955] ? watchdog+0x40/0x40 [ 2712.257371] ? hrtimer_interrupt+0x128/0x2c0 [ 2712.260938] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2712.264355] ? apic_timer_interrupt+0xf/0x20 [ 2712.267454] [ 2712.268213] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2712.273025] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2712.275729] do_raw_spin_lock+0x177/0x190 [ 2712.278066] _raw_spin_lock+0x21/0x30 [ 2712.279861] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2712.283064] ? do_raw_spin_unlock+0x75/0x190 [ 2712.286950] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2712.290306] ? do_wait_intr+0xf0/0xf0 [ 2712.292191] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2712.301785] mdc_reint+0x2d/0x110 [mdc] [ 2712.310269] mdc_unlink+0x658/0x870 [mdc] [ 2712.311771] lmv_unlink+0x34d/0xbd0 [lmv] [ 2712.315492] ll_unlink+0x24d/0x7f0 [lustre] [ 2712.319123] vfs_unlink+0x202/0x330 [ 2712.320628] do_unlinkat+0x2fa/0x460 [ 2712.321959] __x64_sys_unlink+0x37/0x50 [ 2712.323363] do_syscall_64+0xc1/0x440 [ 2712.325023] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2712.326893] RIP: 0033:0x7f33957fc10b [ 2712.328337] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2712.339139] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2712.347696] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2712.352318] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2712.359647] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2712.364703] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2712.368740] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2716.061343] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2716.067605] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2716.099696] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2716.112279] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2716.119399] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2716.125204] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2716.130328] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2716.144110] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2716.152138] RAX: 0000000000007e35 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2716.157555] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2716.163520] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2716.168250] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2716.181279] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2716.186907] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2716.198543] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2716.209901] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2716.214495] Call Trace: [ 2716.215653] [ 2716.217727] ? show_regs.cold.9+0x22/0x2f [ 2716.219250] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2716.222277] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2716.224387] ? watchdog+0x40/0x40 [ 2716.229712] ? hrtimer_interrupt+0x128/0x2c0 [ 2716.231348] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2716.237698] ? apic_timer_interrupt+0xf/0x20 [ 2716.240177] [ 2716.241142] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2716.244201] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2716.248214] do_raw_spin_lock+0x177/0x190 [ 2716.252199] _raw_spin_lock+0x21/0x30 [ 2716.257709] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2716.263487] process_one_work+0x2c8/0x700 [ 2716.264869] worker_thread+0x296/0x6e0 [ 2716.265925] ? rescuer_thread+0x570/0x570 [ 2716.268918] kthread+0x1d1/0x200 [ 2716.271089] ? set_kthread_struct+0x70/0x70 [ 2716.273146] ret_from_fork+0x1f/0x30 [ 2724.053013] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2724.066534] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2724.116788] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2724.130210] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2724.139989] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2724.146809] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2724.165761] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2724.174363] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2724.179347] RDX: 00000000000011ee RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2724.183915] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2724.188706] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2724.193224] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2724.198360] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2724.202197] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2724.207038] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2724.212151] Call Trace: [ 2724.213899] [ 2724.214916] ? show_regs.cold.9+0x22/0x2f [ 2724.218160] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2724.221941] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2724.225338] ? watchdog+0x40/0x40 [ 2724.227381] ? hrtimer_interrupt+0x128/0x2c0 [ 2724.229856] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2724.232274] ? apic_timer_interrupt+0xf/0x20 [ 2724.234735] [ 2724.237103] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2724.240613] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2724.244305] do_raw_spin_lock+0x177/0x190 [ 2724.247353] _raw_spin_lock+0x21/0x30 [ 2724.250826] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2724.254230] obd_mod_free+0x46/0x140 [ptlrpc] [ 2724.258014] mdc_commit_open+0x91/0xb0 [mdc] [ 2724.259920] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2724.263098] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2724.266076] after_reply+0xe2c/0x1780 [ptlrpc] [ 2724.270304] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2724.274522] ? __next_timer_interrupt+0x160/0x160 [ 2724.277456] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2724.280030] ? do_wait_intr+0xf0/0xf0 [ 2724.282307] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2724.285492] kthread+0x1d1/0x200 [ 2724.287223] ? set_kthread_struct+0x70/0x70 [ 2724.289751] ret_from_fork+0x1f/0x30 [ 2740.060443] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2740.068558] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2740.100976] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2740.105294] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2740.108848] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2740.111175] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2740.118574] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2740.121700] RAX: 0000000000007481 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2740.124486] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2740.127232] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2740.130808] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2740.136084] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2740.141428] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2740.147553] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2740.151935] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2740.164459] Call Trace: [ 2740.165741] [ 2740.166653] ? show_regs.cold.9+0x22/0x2f [ 2740.169530] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2740.172797] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2740.175415] ? watchdog+0x40/0x40 [ 2740.177882] ? hrtimer_interrupt+0x128/0x2c0 [ 2740.179672] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2740.183159] ? apic_timer_interrupt+0xf/0x20 [ 2740.185384] [ 2740.187230] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2740.191221] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2740.194037] do_raw_spin_lock+0x177/0x190 [ 2740.197572] _raw_spin_lock+0x21/0x30 [ 2740.199396] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2740.203056] ? do_raw_spin_unlock+0x75/0x190 [ 2740.205258] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2740.208542] ? do_wait_intr+0xf0/0xf0 [ 2740.211221] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2740.214827] mdc_reint+0x2d/0x110 [mdc] [ 2740.218814] mdc_unlink+0x658/0x870 [mdc] [ 2740.222807] lmv_unlink+0x34d/0xbd0 [lmv] [ 2740.224877] ll_unlink+0x24d/0x7f0 [lustre] [ 2740.227719] vfs_unlink+0x202/0x330 [ 2740.229518] do_unlinkat+0x2fa/0x460 [ 2740.231355] __x64_sys_unlink+0x37/0x50 [ 2740.232721] do_syscall_64+0xc1/0x440 [ 2740.234659] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2740.239824] RIP: 0033:0x7f33957fc10b [ 2740.242385] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2740.253143] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2740.257371] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2740.261169] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2740.264855] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2740.269756] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2740.273848] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2744.058026] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2744.066546] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2744.105357] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2744.112312] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2744.119192] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2744.124922] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2744.133912] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2744.150596] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2744.158631] RAX: 0000000000006b0d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2744.169120] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2744.177319] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2744.181948] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2744.186248] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2744.189746] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2744.194959] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2744.198403] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2744.204105] Call Trace: [ 2744.205618] [ 2744.206847] ? show_regs.cold.9+0x22/0x2f [ 2744.209251] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2744.212082] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2744.215087] ? watchdog+0x40/0x40 [ 2744.219702] ? hrtimer_interrupt+0x128/0x2c0 [ 2744.222486] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2744.225337] ? apic_timer_interrupt+0xf/0x20 [ 2744.227637] [ 2744.229039] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2744.231926] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2744.234331] do_raw_spin_lock+0x177/0x190 [ 2744.236361] _raw_spin_lock+0x21/0x30 [ 2744.241610] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2744.246962] process_one_work+0x2c8/0x700 [ 2744.248938] worker_thread+0x296/0x6e0 [ 2744.253213] ? rescuer_thread+0x570/0x570 [ 2744.255049] kthread+0x1d1/0x200 [ 2744.256461] ? set_kthread_struct+0x70/0x70 [ 2744.258040] ret_from_fork+0x1f/0x30 [ 2752.053034] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2752.056126] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2752.080385] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2752.084780] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2752.088315] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2752.090261] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2752.098020] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2752.102819] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2752.105989] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2752.110755] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2752.114682] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2752.118655] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2752.121542] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2752.125857] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2752.129923] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2752.133333] Call Trace: [ 2752.134254] [ 2752.136002] ? show_regs.cold.9+0x22/0x2f [ 2752.138270] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2752.140763] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2752.142798] ? watchdog+0x40/0x40 [ 2752.144581] ? hrtimer_interrupt+0x128/0x2c0 [ 2752.146045] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2752.148557] ? apic_timer_interrupt+0xf/0x20 [ 2752.150177] [ 2752.151189] ? native_safe_halt+0x2e/0x50 [ 2752.153114] kvm_wait+0xcd/0xe0 [ 2752.154635] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2752.157147] do_raw_spin_lock+0x177/0x190 [ 2752.158945] _raw_spin_lock+0x21/0x30 [ 2752.161022] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2752.164765] obd_mod_free+0x46/0x140 [ptlrpc] [ 2752.171264] mdc_commit_open+0x91/0xb0 [mdc] [ 2752.173391] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2752.181210] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2752.183962] after_reply+0xe2c/0x1780 [ptlrpc] [ 2752.187194] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2752.189854] ? __next_timer_interrupt+0x160/0x160 [ 2752.193883] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2752.196781] ? do_wait_intr+0xf0/0xf0 [ 2752.198648] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2752.201594] kthread+0x1d1/0x200 [ 2752.202880] ? set_kthread_struct+0x70/0x70 [ 2752.204879] ret_from_fork+0x1f/0x30 [ 2768.059164] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2768.062739] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2768.087841] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2768.092636] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2768.097330] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2768.099823] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2768.108321] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2768.112261] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2768.116020] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2768.120095] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2768.123404] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2768.127013] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2768.130226] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2768.134438] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2768.137374] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2768.141331] Call Trace: [ 2768.142970] [ 2768.144085] ? show_regs.cold.9+0x22/0x2f [ 2768.145878] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2768.148572] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2768.151450] ? watchdog+0x40/0x40 [ 2768.153320] ? hrtimer_interrupt+0x128/0x2c0 [ 2768.155672] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2768.158279] ? apic_timer_interrupt+0xf/0x20 [ 2768.160383] [ 2768.162340] ? native_safe_halt+0x2e/0x50 [ 2768.164315] kvm_wait+0xcd/0xe0 [ 2768.166414] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2768.169314] do_raw_spin_lock+0x177/0x190 [ 2768.171585] _raw_spin_lock+0x21/0x30 [ 2768.173471] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2768.176676] ? do_raw_spin_unlock+0x75/0x190 [ 2768.179320] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2768.183396] ? do_wait_intr+0xf0/0xf0 [ 2768.185476] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2768.188585] mdc_reint+0x2d/0x110 [mdc] [ 2768.191122] mdc_unlink+0x658/0x870 [mdc] [ 2768.194079] lmv_unlink+0x34d/0xbd0 [lmv] [ 2768.196128] ll_unlink+0x24d/0x7f0 [lustre] [ 2768.199117] vfs_unlink+0x202/0x330 [ 2768.201118] do_unlinkat+0x2fa/0x460 [ 2768.203305] __x64_sys_unlink+0x37/0x50 [ 2768.205194] do_syscall_64+0xc1/0x440 [ 2768.207373] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2768.210280] RIP: 0033:0x7f33957fc10b [ 2768.212489] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2768.224066] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2768.228413] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2768.232282] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2768.236253] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2768.240194] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2768.244067] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2772.059593] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2772.067152] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2772.109077] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2772.119975] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2772.126516] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2772.132304] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2772.145186] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2772.169482] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2772.176126] RAX: 0000000000007e5e RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2772.180007] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2772.188724] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2772.198070] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2772.205535] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2772.213949] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2772.226107] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2772.234848] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2772.242536] Call Trace: [ 2772.244999] [ 2772.246359] ? show_regs.cold.9+0x22/0x2f [ 2772.249126] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2772.252174] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2772.256077] ? watchdog+0x40/0x40 [ 2772.258386] ? hrtimer_interrupt+0x128/0x2c0 [ 2772.266806] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2772.271434] ? apic_timer_interrupt+0xf/0x20 [ 2772.275438] [ 2772.281116] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2772.289998] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2772.294899] do_raw_spin_lock+0x177/0x190 [ 2772.300373] _raw_spin_lock+0x21/0x30 [ 2772.303400] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2772.308584] process_one_work+0x2c8/0x700 [ 2772.314192] worker_thread+0x296/0x6e0 [ 2772.317542] ? rescuer_thread+0x570/0x570 [ 2772.322073] kthread+0x1d1/0x200 [ 2772.327614] ? set_kthread_struct+0x70/0x70 [ 2772.331583] ret_from_fork+0x1f/0x30 [ 2780.053056] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2780.063473] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2780.102374] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2780.116145] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2780.120295] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2780.126390] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2780.137358] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2780.141746] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2780.145212] RDX: 00000000000052fb RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2780.150623] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2780.153689] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2780.158405] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2780.169693] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2780.177700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2780.187242] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2780.193952] Call Trace: [ 2780.197776] [ 2780.198946] ? show_regs.cold.9+0x22/0x2f [ 2780.205180] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2780.210324] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2780.213919] ? watchdog+0x40/0x40 [ 2780.215435] ? hrtimer_interrupt+0x128/0x2c0 [ 2780.219896] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2780.222870] ? apic_timer_interrupt+0xf/0x20 [ 2780.231473] [ 2780.232582] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2780.239414] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2780.242688] do_raw_spin_lock+0x177/0x190 [ 2780.245791] _raw_spin_lock+0x21/0x30 [ 2780.248202] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2780.253002] obd_mod_free+0x46/0x140 [ptlrpc] [ 2780.256419] mdc_commit_open+0x91/0xb0 [mdc] [ 2780.259154] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2780.262411] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2780.267259] after_reply+0xe2c/0x1780 [ptlrpc] [ 2780.274418] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2780.284271] ? __next_timer_interrupt+0x160/0x160 [ 2780.290338] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2780.295169] ? do_wait_intr+0xf0/0xf0 [ 2780.299107] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2780.304077] kthread+0x1d1/0x200 [ 2780.305323] ? set_kthread_struct+0x70/0x70 [ 2780.307755] ret_from_fork+0x1f/0x30 [ 2796.063182] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2796.070146] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2796.106352] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2796.113229] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2796.117732] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2796.123411] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2796.143587] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2796.151361] RAX: 0000000000005394 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2796.155871] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2796.163412] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2796.169381] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2796.179798] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2796.184753] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2796.191605] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2796.193353] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2796.199368] Call Trace: [ 2796.200765] [ 2796.206286] ? show_regs.cold.9+0x22/0x2f [ 2796.207781] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2796.217722] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2796.221610] ? watchdog+0x40/0x40 [ 2796.223163] ? hrtimer_interrupt+0x128/0x2c0 [ 2796.228322] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2796.230879] ? apic_timer_interrupt+0xf/0x20 [ 2796.238878] [ 2796.239933] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2796.244240] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2796.250811] do_raw_spin_lock+0x177/0x190 [ 2796.253521] _raw_spin_lock+0x21/0x30 [ 2796.261329] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2796.264783] ? do_raw_spin_unlock+0x75/0x190 [ 2796.267325] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2796.269355] ? do_wait_intr+0xf0/0xf0 [ 2796.271038] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2796.277725] mdc_reint+0x2d/0x110 [mdc] [ 2796.284725] mdc_unlink+0x658/0x870 [mdc] [ 2796.289012] lmv_unlink+0x34d/0xbd0 [lmv] [ 2796.291500] ll_unlink+0x24d/0x7f0 [lustre] [ 2796.294838] vfs_unlink+0x202/0x330 [ 2796.300048] do_unlinkat+0x2fa/0x460 [ 2796.305355] __x64_sys_unlink+0x37/0x50 [ 2796.306732] do_syscall_64+0xc1/0x440 [ 2796.310391] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2796.312721] RIP: 0033:0x7f33957fc10b [ 2796.323551] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2796.340997] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2796.347440] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2796.352711] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2796.359686] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2796.365088] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2796.371152] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2800.058242] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2800.068104] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2800.091898] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2800.096902] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2800.102092] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2800.105949] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2800.108295] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2800.117212] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2800.120554] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2800.124231] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2800.127217] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2800.130073] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2800.133079] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2800.135559] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2800.138799] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2800.141060] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2800.143769] Call Trace: [ 2800.144616] [ 2800.145486] ? show_regs.cold.9+0x22/0x2f [ 2800.147014] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2800.149862] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2800.151824] ? watchdog+0x40/0x40 [ 2800.153523] ? hrtimer_interrupt+0x128/0x2c0 [ 2800.155662] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2800.158517] ? apic_timer_interrupt+0xf/0x20 [ 2800.161050] [ 2800.162749] ? native_safe_halt+0x2e/0x50 [ 2800.165275] kvm_wait+0xcd/0xe0 [ 2800.167322] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2800.169994] do_raw_spin_lock+0x177/0x190 [ 2800.172429] _raw_spin_lock+0x21/0x30 [ 2800.174468] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2800.177587] process_one_work+0x2c8/0x700 [ 2800.180314] worker_thread+0x296/0x6e0 [ 2800.183032] ? rescuer_thread+0x570/0x570 [ 2800.185518] kthread+0x1d1/0x200 [ 2800.188172] ? set_kthread_struct+0x70/0x70 [ 2800.195159] ret_from_fork+0x1f/0x30 [ 2808.055546] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2808.064760] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2808.126016] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2808.139597] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2808.152811] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2808.162687] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2808.187646] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2808.206137] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2808.214580] RDX: 0000000000006b0f RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2808.224159] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2808.235188] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2808.248325] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2808.260516] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2808.270480] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2808.275003] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2808.277189] Call Trace: [ 2808.280481] [ 2808.281430] ? show_regs.cold.9+0x22/0x2f [ 2808.284395] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2808.286607] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2808.293972] ? watchdog+0x40/0x40 [ 2808.295458] ? hrtimer_interrupt+0x128/0x2c0 [ 2808.301606] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2808.303163] ? apic_timer_interrupt+0xf/0x20 [ 2808.304632] [ 2808.306102] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2808.308464] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2808.311665] do_raw_spin_lock+0x177/0x190 [ 2808.314462] _raw_spin_lock+0x21/0x30 [ 2808.320206] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2808.323530] obd_mod_free+0x46/0x140 [ptlrpc] [ 2808.327688] mdc_commit_open+0x91/0xb0 [mdc] [ 2808.332320] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2808.335107] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2808.338472] after_reply+0xe2c/0x1780 [ptlrpc] [ 2808.344596] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2808.346507] ? __next_timer_interrupt+0x160/0x160 [ 2808.353594] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2808.356367] ? do_wait_intr+0xf0/0xf0 [ 2808.358666] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2808.363615] kthread+0x1d1/0x200 [ 2808.368234] ? set_kthread_struct+0x70/0x70 [ 2808.370006] ret_from_fork+0x1f/0x30 [ 2824.061770] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2824.071675] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2824.131332] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2824.154896] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2824.164222] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2824.181538] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2824.207522] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2824.218007] RAX: 0000000000007989 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2824.221019] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2824.223615] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2824.228137] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2824.237913] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2824.241441] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2824.245129] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2824.247820] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2824.249963] Call Trace: [ 2824.250750] [ 2824.251429] ? show_regs.cold.9+0x22/0x2f [ 2824.252580] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2824.254088] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2824.255563] ? watchdog+0x40/0x40 [ 2824.256464] ? hrtimer_interrupt+0x128/0x2c0 [ 2824.258425] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2824.260751] ? apic_timer_interrupt+0xf/0x20 [ 2824.262838] [ 2824.263919] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2824.266036] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2824.268517] do_raw_spin_lock+0x177/0x190 [ 2824.270582] _raw_spin_lock+0x21/0x30 [ 2824.272487] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2824.275449] ? do_raw_spin_unlock+0x75/0x190 [ 2824.277341] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2824.279641] ? do_wait_intr+0xf0/0xf0 [ 2824.281941] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2824.283951] mdc_reint+0x2d/0x110 [mdc] [ 2824.286946] mdc_unlink+0x658/0x870 [mdc] [ 2824.291486] lmv_unlink+0x34d/0xbd0 [lmv] [ 2824.296469] ll_unlink+0x24d/0x7f0 [lustre] [ 2824.297890] vfs_unlink+0x202/0x330 [ 2824.299935] do_unlinkat+0x2fa/0x460 [ 2824.302103] __x64_sys_unlink+0x37/0x50 [ 2824.303721] do_syscall_64+0xc1/0x440 [ 2824.304913] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2824.306829] RIP: 0033:0x7f33957fc10b [ 2824.308451] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2824.315997] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2824.318499] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2824.321483] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2824.323820] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2824.326329] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2824.328740] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2828.058031] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2828.064930] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2828.116957] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2828.131038] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2828.143974] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2828.153845] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2828.162801] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2828.189059] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2828.212069] RAX: 00000000000032b5 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2828.214578] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2828.222812] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2828.227764] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2828.234849] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2828.239851] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2828.246971] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2828.254133] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2828.262823] Call Trace: [ 2828.265853] [ 2828.268390] ? show_regs.cold.9+0x22/0x2f [ 2828.272843] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2828.282675] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2828.285895] ? watchdog+0x40/0x40 [ 2828.287437] ? hrtimer_interrupt+0x128/0x2c0 [ 2828.292565] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2828.294706] ? apic_timer_interrupt+0xf/0x20 [ 2828.297873] [ 2828.299110] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2828.302395] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2828.306558] do_raw_spin_lock+0x177/0x190 [ 2828.313105] _raw_spin_lock+0x21/0x30 [ 2828.316955] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2828.320514] process_one_work+0x2c8/0x700 [ 2828.322811] worker_thread+0x296/0x6e0 [ 2828.330416] ? rescuer_thread+0x570/0x570 [ 2828.334051] kthread+0x1d1/0x200 [ 2828.339108] ? set_kthread_struct+0x70/0x70 [ 2828.341074] ret_from_fork+0x1f/0x30 [ 2836.054282] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 2836.061508] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2836.090076] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2836.101628] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2836.109003] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2836.113242] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2836.125252] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2836.131891] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2836.140219] RDX: 0000000000000f3a RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2836.147866] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2836.153026] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2836.159878] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2836.166927] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2836.177989] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2836.181178] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2836.185564] Call Trace: [ 2836.188858] [ 2836.190562] ? show_regs.cold.9+0x22/0x2f [ 2836.193775] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2836.196255] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2836.198597] ? watchdog+0x40/0x40 [ 2836.201004] ? hrtimer_interrupt+0x128/0x2c0 [ 2836.206700] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2836.211719] ? apic_timer_interrupt+0xf/0x20 [ 2836.214962] [ 2836.219206] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2836.221678] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2836.227604] do_raw_spin_lock+0x177/0x190 [ 2836.232960] _raw_spin_lock+0x21/0x30 [ 2836.234746] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2836.241386] obd_mod_free+0x46/0x140 [ptlrpc] [ 2836.246229] mdc_commit_open+0x91/0xb0 [mdc] [ 2836.252489] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2836.255225] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2836.263415] after_reply+0xe2c/0x1780 [ptlrpc] [ 2836.267193] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2836.273073] ? __next_timer_interrupt+0x160/0x160 [ 2836.277724] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2836.282373] ? do_wait_intr+0xf0/0xf0 [ 2836.283993] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2836.289897] kthread+0x1d1/0x200 [ 2836.292198] ? set_kthread_struct+0x70/0x70 [ 2836.297421] ret_from_fork+0x1f/0x30 [ 2852.061231] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2852.074550] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2852.145973] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2852.153807] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2852.171659] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2852.177746] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2852.199574] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2852.218215] RAX: 000000000000448d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2852.229494] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2852.238498] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2852.241957] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2852.250935] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2852.254877] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2852.265058] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2852.277980] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2852.285696] Call Trace: [ 2852.287556] [ 2852.290099] ? show_regs.cold.9+0x22/0x2f [ 2852.291495] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2852.298943] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2852.305494] ? watchdog+0x40/0x40 [ 2852.307207] ? hrtimer_interrupt+0x128/0x2c0 [ 2852.309581] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2852.312147] ? apic_timer_interrupt+0xf/0x20 [ 2852.314163] [ 2852.315490] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2852.317613] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2852.324182] do_raw_spin_lock+0x177/0x190 [ 2852.326754] _raw_spin_lock+0x21/0x30 [ 2852.330655] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2852.336490] ? do_raw_spin_unlock+0x75/0x190 [ 2852.350458] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2852.356538] ? do_wait_intr+0xf0/0xf0 [ 2852.358945] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2852.368226] mdc_reint+0x2d/0x110 [mdc] [ 2852.370991] mdc_unlink+0x658/0x870 [mdc] [ 2852.375000] lmv_unlink+0x34d/0xbd0 [lmv] [ 2852.381296] ll_unlink+0x24d/0x7f0 [lustre] [ 2852.387806] vfs_unlink+0x202/0x330 [ 2852.392726] do_unlinkat+0x2fa/0x460 [ 2852.394794] __x64_sys_unlink+0x37/0x50 [ 2852.397901] do_syscall_64+0xc1/0x440 [ 2852.403456] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2852.410834] RIP: 0033:0x7f33957fc10b [ 2852.414701] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2852.432787] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2852.441366] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2852.447796] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2852.453920] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2852.462471] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2852.473054] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2856.058319] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2856.075504] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2856.102194] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2856.109241] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2856.114714] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2856.118481] RIP: 0010:__pv_queued_spin_lock_slowpath+0x328/0x6d0 [ 2856.122024] Code: d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 83 e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 05 18 d4 dc 02 01 e0 48 83 05 06 d4 dc 02 01 48 83 05 be d5 dc 02 01 8b 45 08 85 [ 2856.132176] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2856.136463] RAX: 000000000000487b RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2856.140708] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2856.144275] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2856.148402] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2856.152379] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2856.157088] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2856.161964] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2856.164832] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2856.167249] Call Trace: [ 2856.168049] [ 2856.169124] ? show_regs.cold.9+0x22/0x2f [ 2856.171758] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2856.174398] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2856.176761] ? watchdog+0x40/0x40 [ 2856.177969] ? hrtimer_interrupt+0x128/0x2c0 [ 2856.179968] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2856.182089] ? apic_timer_interrupt+0xf/0x20 [ 2856.184891] [ 2856.186604] ? __pv_queued_spin_lock_slowpath+0x328/0x6d0 [ 2856.191115] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2856.195677] do_raw_spin_lock+0x177/0x190 [ 2856.198144] _raw_spin_lock+0x21/0x30 [ 2856.199518] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2856.203582] process_one_work+0x2c8/0x700 [ 2856.209963] worker_thread+0x296/0x6e0 [ 2856.213368] ? rescuer_thread+0x570/0x570 [ 2856.218255] kthread+0x1d1/0x200 [ 2856.222161] ? set_kthread_struct+0x70/0x70 [ 2856.226415] ret_from_fork+0x1f/0x30 [ 2864.053055] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2864.057563] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2864.079397] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2864.083696] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2864.087732] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2864.091308] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2864.105161] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2864.111351] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2864.114334] RDX: 000000000000332f RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2864.116716] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2864.119503] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2864.121984] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2864.125922] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2864.129156] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2864.131312] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2864.134216] Call Trace: [ 2864.135197] [ 2864.136128] ? show_regs.cold.9+0x22/0x2f [ 2864.137674] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2864.139360] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2864.141329] ? watchdog+0x40/0x40 [ 2864.143299] ? hrtimer_interrupt+0x128/0x2c0 [ 2864.144763] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2864.147688] ? apic_timer_interrupt+0xf/0x20 [ 2864.150391] [ 2864.151201] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2864.153230] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2864.156101] do_raw_spin_lock+0x177/0x190 [ 2864.157854] _raw_spin_lock+0x21/0x30 [ 2864.159476] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2864.161759] obd_mod_free+0x46/0x140 [ptlrpc] [ 2864.163519] mdc_commit_open+0x91/0xb0 [mdc] [ 2864.165176] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2864.167726] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2864.170290] after_reply+0xe2c/0x1780 [ptlrpc] [ 2864.172756] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2864.175307] ? __next_timer_interrupt+0x160/0x160 [ 2864.177082] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2864.178927] ? do_wait_intr+0xf0/0xf0 [ 2864.180452] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2864.183616] kthread+0x1d1/0x200 [ 2864.188001] ? set_kthread_struct+0x70/0x70 [ 2864.192173] ret_from_fork+0x1f/0x30 [ 2878.038120] rcu: INFO: rcu_sched self-detected stall on CPU [ 2878.050394] rcu: 0-....: (466027 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=143937 [ 2878.063147] (t=780018 jiffies g=297885 q=5835) [ 2878.074071] NMI backtrace for cpu 0 [ 2878.083409] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2878.100447] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2878.115326] Call Trace: [ 2878.120158] [ 2878.124979] ? dump_stack+0xbb/0x10e [ 2878.129919] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 2878.139027] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 2878.144040] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 2878.145989] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 2878.161259] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 2878.167274] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 2878.172027] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 2878.175945] ? update_process_times+0x6d/0xc0 [ 2878.179278] ? tick_sched_handle.isra.18+0x27/0x90 [ 2878.183421] ? tick_sched_timer+0x53/0xb0 [ 2878.189355] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2878.193303] ? tick_sched_handle.isra.18+0x90/0x90 [ 2878.200396] ? hrtimer_interrupt+0x128/0x2c0 [ 2878.206653] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2878.208482] ? apic_timer_interrupt+0xf/0x20 [ 2878.213257] [ 2878.216286] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2878.230046] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2878.244265] ? do_raw_spin_lock+0x177/0x190 [ 2878.253295] ? _raw_spin_lock+0x21/0x30 [ 2878.254788] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2878.263561] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 2878.273349] ? mdc_commit_open+0x91/0xb0 [mdc] [ 2878.275849] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2878.279448] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2878.284611] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 2878.292335] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2878.302537] ? __next_timer_interrupt+0x160/0x160 [ 2878.305591] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2878.309409] ? do_wait_intr+0xf0/0xf0 [ 2878.311963] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2878.314253] ? kthread+0x1d1/0x200 [ 2878.318334] ? set_kthread_struct+0x70/0x70 [ 2878.321394] ? ret_from_fork+0x1f/0x30 [ 2880.064891] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2880.075356] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2880.132968] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2880.140994] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2880.154305] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2880.160779] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2880.192209] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2880.199836] RAX: 000000000000059d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2880.205840] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2880.212986] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2880.218732] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2880.228458] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2880.247995] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2880.255324] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2880.260735] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2880.266269] Call Trace: [ 2880.267975] [ 2880.269043] ? show_regs.cold.9+0x22/0x2f [ 2880.271493] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2880.276878] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2880.282640] ? watchdog+0x40/0x40 [ 2880.284251] ? hrtimer_interrupt+0x128/0x2c0 [ 2880.287082] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2880.294964] ? apic_timer_interrupt+0xf/0x20 [ 2880.298164] [ 2880.299070] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2880.303648] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2880.308751] do_raw_spin_lock+0x177/0x190 [ 2880.317466] _raw_spin_lock+0x21/0x30 [ 2880.322408] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2880.329425] ? do_raw_spin_unlock+0x75/0x190 [ 2880.333845] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2880.340070] ? do_wait_intr+0xf0/0xf0 [ 2880.345131] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2880.350385] mdc_reint+0x2d/0x110 [mdc] [ 2880.355125] mdc_unlink+0x658/0x870 [mdc] [ 2880.358365] lmv_unlink+0x34d/0xbd0 [lmv] [ 2880.361886] ll_unlink+0x24d/0x7f0 [lustre] [ 2880.367369] vfs_unlink+0x202/0x330 [ 2880.374991] do_unlinkat+0x2fa/0x460 [ 2880.381132] __x64_sys_unlink+0x37/0x50 [ 2880.395195] do_syscall_64+0xc1/0x440 [ 2880.400499] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2880.404763] RIP: 0033:0x7f33957fc10b [ 2880.408287] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2880.440357] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2880.449897] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2880.457304] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2880.466932] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2880.477810] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2880.486548] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2884.058068] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 2884.072553] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2884.147949] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2884.158757] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2884.170854] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2884.178389] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2884.181827] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2884.191376] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2884.195236] RAX: 0000000000001b14 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2884.198367] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2884.202297] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2884.205938] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2884.211466] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2884.214841] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2884.219751] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2884.222838] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2884.227291] Call Trace: [ 2884.228689] [ 2884.229830] ? show_regs.cold.9+0x22/0x2f [ 2884.232627] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2884.235230] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2884.237852] ? watchdog+0x40/0x40 [ 2884.239762] ? hrtimer_interrupt+0x128/0x2c0 [ 2884.242267] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2884.244818] ? apic_timer_interrupt+0xf/0x20 [ 2884.247280] [ 2884.248363] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2884.251743] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2884.254280] do_raw_spin_lock+0x177/0x190 [ 2884.256649] _raw_spin_lock+0x21/0x30 [ 2884.258856] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2884.261742] process_one_work+0x2c8/0x700 [ 2884.264297] worker_thread+0x296/0x6e0 [ 2884.267058] ? rescuer_thread+0x570/0x570 [ 2884.269836] kthread+0x1d1/0x200 [ 2884.271979] ? set_kthread_struct+0x70/0x70 [ 2884.274580] ret_from_fork+0x1f/0x30 [ 2904.058411] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2904.073430] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2904.125308] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2904.144616] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2904.159753] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2904.168543] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2904.183429] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2904.192269] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2904.200737] RDX: 0000000000006fbd RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2904.213794] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2904.220950] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2904.229353] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2904.236442] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2904.247591] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2904.254025] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2904.258346] Call Trace: [ 2904.260608] [ 2904.266341] ? show_regs.cold.9+0x22/0x2f [ 2904.272595] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2904.276595] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2904.279803] ? watchdog+0x40/0x40 [ 2904.282446] ? hrtimer_interrupt+0x128/0x2c0 [ 2904.285404] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2904.288180] ? apic_timer_interrupt+0xf/0x20 [ 2904.290454] [ 2904.291744] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2904.294139] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2904.297194] do_raw_spin_lock+0x177/0x190 [ 2904.299515] _raw_spin_lock+0x21/0x30 [ 2904.301644] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2904.304519] obd_mod_free+0x46/0x140 [ptlrpc] [ 2904.307127] mdc_commit_open+0x91/0xb0 [mdc] [ 2904.309225] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2904.312033] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2904.314612] after_reply+0xe2c/0x1780 [ptlrpc] [ 2904.316533] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2904.319585] ? __next_timer_interrupt+0x160/0x160 [ 2904.322010] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2904.324496] ? do_wait_intr+0xf0/0xf0 [ 2904.325835] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2904.328492] kthread+0x1d1/0x200 [ 2904.330255] ? set_kthread_struct+0x70/0x70 [ 2904.332335] ret_from_fork+0x1f/0x30 [ 2908.059025] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 2908.062591] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2908.088675] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2908.095011] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2908.099746] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2908.102299] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2908.114232] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2908.119337] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2908.123697] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2908.128507] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2908.131061] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2908.135101] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2908.139102] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2908.144149] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2908.147131] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2908.150989] Call Trace: [ 2908.152315] [ 2908.153389] ? show_regs.cold.9+0x22/0x2f [ 2908.156219] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2908.159018] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2908.161254] ? watchdog+0x40/0x40 [ 2908.163023] ? hrtimer_interrupt+0x128/0x2c0 [ 2908.165247] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2908.167807] ? apic_timer_interrupt+0xf/0x20 [ 2908.170260] [ 2908.171251] ? native_safe_halt+0x2e/0x50 [ 2908.173945] kvm_wait+0xcd/0xe0 [ 2908.175377] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2908.178137] do_raw_spin_lock+0x177/0x190 [ 2908.180301] _raw_spin_lock+0x21/0x30 [ 2908.182135] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2908.185328] ? do_raw_spin_unlock+0x75/0x190 [ 2908.187485] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2908.190252] ? do_wait_intr+0xf0/0xf0 [ 2908.192171] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2908.195097] mdc_reint+0x2d/0x110 [mdc] [ 2908.197377] mdc_unlink+0x658/0x870 [mdc] [ 2908.201796] lmv_unlink+0x34d/0xbd0 [lmv] [ 2908.204551] ll_unlink+0x24d/0x7f0 [lustre] [ 2908.206804] vfs_unlink+0x202/0x330 [ 2908.208645] do_unlinkat+0x2fa/0x460 [ 2908.210562] __x64_sys_unlink+0x37/0x50 [ 2908.212786] do_syscall_64+0xc1/0x440 [ 2908.214760] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2908.217676] RIP: 0033:0x7f33957fc10b [ 2908.220020] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2908.230748] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2908.234854] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2908.239204] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2908.244045] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2908.248046] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2908.252220] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2912.058227] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2912.070671] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2912.101662] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2912.107588] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2912.115559] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2912.118999] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2912.124831] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2912.140303] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2912.145975] RAX: 0000000000005305 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2912.149644] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2912.152593] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2912.155785] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2912.159359] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2912.165396] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2912.171580] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2912.179529] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2912.184826] Call Trace: [ 2912.186335] [ 2912.189278] ? show_regs.cold.9+0x22/0x2f [ 2912.195604] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2912.198255] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2912.200852] ? watchdog+0x40/0x40 [ 2912.202330] ? hrtimer_interrupt+0x128/0x2c0 [ 2912.206182] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2912.208423] ? apic_timer_interrupt+0xf/0x20 [ 2912.210581] [ 2912.212055] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2912.216590] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2912.218943] do_raw_spin_lock+0x177/0x190 [ 2912.221393] _raw_spin_lock+0x21/0x30 [ 2912.223260] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2912.226214] process_one_work+0x2c8/0x700 [ 2912.230302] worker_thread+0x296/0x6e0 [ 2912.234136] ? rescuer_thread+0x570/0x570 [ 2912.237420] kthread+0x1d1/0x200 [ 2912.241246] ? set_kthread_struct+0x70/0x70 [ 2912.243089] ret_from_fork+0x1f/0x30 [ 2932.056906] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2932.062033] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2932.084811] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2932.090289] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2932.094421] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2932.097347] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2932.105868] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2932.110029] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2932.113782] RDX: 000000000000204a RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2932.117370] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2932.120797] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2932.124479] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2932.128329] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2932.132887] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2932.140201] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2932.149707] Call Trace: [ 2932.152176] [ 2932.153165] ? show_regs.cold.9+0x22/0x2f [ 2932.157814] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2932.163080] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2932.167121] ? watchdog+0x40/0x40 [ 2932.170498] ? hrtimer_interrupt+0x128/0x2c0 [ 2932.174569] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2932.180090] ? apic_timer_interrupt+0xf/0x20 [ 2932.184536] [ 2932.187084] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2932.192992] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2932.201137] do_raw_spin_lock+0x177/0x190 [ 2932.203952] _raw_spin_lock+0x21/0x30 [ 2932.208610] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2932.215369] obd_mod_free+0x46/0x140 [ptlrpc] [ 2932.221833] mdc_commit_open+0x91/0xb0 [mdc] [ 2932.225297] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2932.232645] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2932.240902] after_reply+0xe2c/0x1780 [ptlrpc] [ 2932.247352] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2932.255531] ? __next_timer_interrupt+0x160/0x160 [ 2932.259702] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2932.262324] ? do_wait_intr+0xf0/0xf0 [ 2932.264247] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2932.267023] kthread+0x1d1/0x200 [ 2932.269152] ? set_kthread_struct+0x70/0x70 [ 2932.271328] ret_from_fork+0x1f/0x30 [ 2936.059074] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2936.063811] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2936.115750] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2936.123329] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2936.132582] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2936.141212] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2936.155637] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2936.162943] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2936.168501] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2936.178634] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2936.184865] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2936.194681] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2936.204594] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2936.215192] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2936.220850] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2936.231567] Call Trace: [ 2936.233780] [ 2936.235332] ? show_regs.cold.9+0x22/0x2f [ 2936.241564] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2936.248797] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2936.256618] ? watchdog+0x40/0x40 [ 2936.259537] ? hrtimer_interrupt+0x128/0x2c0 [ 2936.267974] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2936.271869] ? apic_timer_interrupt+0xf/0x20 [ 2936.275826] [ 2936.278236] ? native_safe_halt+0x2e/0x50 [ 2936.280358] kvm_wait+0xcd/0xe0 [ 2936.281510] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2936.291620] do_raw_spin_lock+0x177/0x190 [ 2936.295654] _raw_spin_lock+0x21/0x30 [ 2936.299962] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2936.303913] ? do_raw_spin_unlock+0x75/0x190 [ 2936.309040] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2936.311812] ? do_wait_intr+0xf0/0xf0 [ 2936.320403] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2936.325399] mdc_reint+0x2d/0x110 [mdc] [ 2936.327811] mdc_unlink+0x658/0x870 [mdc] [ 2936.330815] lmv_unlink+0x34d/0xbd0 [lmv] [ 2936.333275] ll_unlink+0x24d/0x7f0 [lustre] [ 2936.340748] vfs_unlink+0x202/0x330 [ 2936.342017] do_unlinkat+0x2fa/0x460 [ 2936.345725] __x64_sys_unlink+0x37/0x50 [ 2936.347667] do_syscall_64+0xc1/0x440 [ 2936.351872] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2936.356112] RIP: 0033:0x7f33957fc10b [ 2936.358859] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2936.367239] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2936.370199] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2936.373771] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2936.378130] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2936.380958] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2936.387232] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2940.058550] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2940.062740] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2940.097556] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2940.106681] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2940.114644] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2940.119643] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2940.122892] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2940.138301] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2940.143207] RAX: 0000000000003a18 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2940.147836] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2940.154229] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2940.158298] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2940.161448] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2940.165118] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2940.169798] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2940.177299] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2940.181564] Call Trace: [ 2940.182695] [ 2940.183843] ? show_regs.cold.9+0x22/0x2f [ 2940.187717] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2940.192085] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2940.195943] ? watchdog+0x40/0x40 [ 2940.200305] ? hrtimer_interrupt+0x128/0x2c0 [ 2940.202674] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2940.206890] ? apic_timer_interrupt+0xf/0x20 [ 2940.212393] [ 2940.214774] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2940.219397] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2940.227795] do_raw_spin_lock+0x177/0x190 [ 2940.236986] _raw_spin_lock+0x21/0x30 [ 2940.239726] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2940.248837] process_one_work+0x2c8/0x700 [ 2940.251266] worker_thread+0x296/0x6e0 [ 2940.255895] ? rescuer_thread+0x570/0x570 [ 2940.258127] kthread+0x1d1/0x200 [ 2940.261810] ? set_kthread_struct+0x70/0x70 [ 2940.264164] ret_from_fork+0x1f/0x30 [ 2960.053422] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2960.060879] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2960.088962] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2960.094211] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2960.098667] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2960.101498] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2960.111575] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2960.114625] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2960.117694] RDX: 0000000000007061 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2960.120679] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2960.124333] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2960.128152] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2960.132054] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2960.136175] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2960.138562] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2960.143018] Call Trace: [ 2960.144291] [ 2960.145467] ? show_regs.cold.9+0x22/0x2f [ 2960.147427] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2960.149809] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2960.152147] ? watchdog+0x40/0x40 [ 2960.154074] ? hrtimer_interrupt+0x128/0x2c0 [ 2960.156716] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2960.158736] ? apic_timer_interrupt+0xf/0x20 [ 2960.162102] [ 2960.163148] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2960.165558] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2960.169080] do_raw_spin_lock+0x177/0x190 [ 2960.170772] _raw_spin_lock+0x21/0x30 [ 2960.173408] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2960.176331] obd_mod_free+0x46/0x140 [ptlrpc] [ 2960.179080] mdc_commit_open+0x91/0xb0 [mdc] [ 2960.182022] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2960.184636] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2960.187916] after_reply+0xe2c/0x1780 [ptlrpc] [ 2960.191203] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2960.194283] ? __next_timer_interrupt+0x160/0x160 [ 2960.197096] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2960.200136] ? do_wait_intr+0xf0/0xf0 [ 2960.202568] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2960.205322] kthread+0x1d1/0x200 [ 2960.207738] ? set_kthread_struct+0x70/0x70 [ 2960.210965] ret_from_fork+0x1f/0x30 [ 2964.059587] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2964.063161] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2964.114079] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2964.120943] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2964.123611] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2964.126816] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2964.138629] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 2964.143541] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2964.148494] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2964.154377] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2964.159456] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2964.165707] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2964.169980] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2964.173337] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2964.176064] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2964.179802] Call Trace: [ 2964.181069] [ 2964.182130] ? show_regs.cold.9+0x22/0x2f [ 2964.184122] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2964.186818] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2964.189647] ? watchdog+0x40/0x40 [ 2964.191823] ? hrtimer_interrupt+0x128/0x2c0 [ 2964.195096] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2964.199742] ? apic_timer_interrupt+0xf/0x20 [ 2964.202355] [ 2964.203603] ? native_safe_halt+0x2e/0x50 [ 2964.206112] kvm_wait+0xcd/0xe0 [ 2964.207668] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2964.210104] do_raw_spin_lock+0x177/0x190 [ 2964.213068] _raw_spin_lock+0x21/0x30 [ 2964.215356] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2964.219383] ? do_raw_spin_unlock+0x75/0x190 [ 2964.221548] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2964.224303] ? do_wait_intr+0xf0/0xf0 [ 2964.225231] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2964.228086] mdc_reint+0x2d/0x110 [mdc] [ 2964.230640] mdc_unlink+0x658/0x870 [mdc] [ 2964.233206] lmv_unlink+0x34d/0xbd0 [lmv] [ 2964.235144] ll_unlink+0x24d/0x7f0 [lustre] [ 2964.237570] vfs_unlink+0x202/0x330 [ 2964.239734] do_unlinkat+0x2fa/0x460 [ 2964.241835] __x64_sys_unlink+0x37/0x50 [ 2964.244128] do_syscall_64+0xc1/0x440 [ 2964.246335] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2964.251047] RIP: 0033:0x7f33957fc10b [ 2964.252652] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2964.264412] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2964.267606] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2964.272621] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2964.276271] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2964.280589] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2964.283834] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2968.058083] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2968.068818] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2968.117957] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2968.138888] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2968.145741] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2968.150532] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2968.157842] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 2968.183296] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 2968.190763] RAX: 000000000000779c RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2968.199046] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2968.216183] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2968.225414] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2968.236785] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2968.244343] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2968.251941] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2968.258800] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2968.268183] Call Trace: [ 2968.270261] [ 2968.272329] ? show_regs.cold.9+0x22/0x2f [ 2968.277612] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2968.281213] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2968.284060] ? watchdog+0x40/0x40 [ 2968.286693] ? hrtimer_interrupt+0x128/0x2c0 [ 2968.293980] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2968.298565] ? apic_timer_interrupt+0xf/0x20 [ 2968.304620] [ 2968.305758] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 2968.309488] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2968.315984] do_raw_spin_lock+0x177/0x190 [ 2968.319429] _raw_spin_lock+0x21/0x30 [ 2968.322163] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2968.330739] process_one_work+0x2c8/0x700 [ 2968.334604] worker_thread+0x296/0x6e0 [ 2968.337353] ? rescuer_thread+0x570/0x570 [ 2968.342989] kthread+0x1d1/0x200 [ 2968.344465] ? set_kthread_struct+0x70/0x70 [ 2968.352214] ret_from_fork+0x1f/0x30 [ 2988.053437] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 2988.057057] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2988.079112] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2988.084210] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2988.087621] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2988.090112] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 2988.098858] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2988.102742] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 2988.107348] RDX: 000000000000423c RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 2988.111286] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 2988.115084] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 2988.119530] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 2988.122454] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 2988.126338] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2988.129266] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 2988.132577] Call Trace: [ 2988.133859] [ 2988.134854] ? show_regs.cold.9+0x22/0x2f [ 2988.137399] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2988.139718] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2988.142275] ? watchdog+0x40/0x40 [ 2988.144141] ? hrtimer_interrupt+0x128/0x2c0 [ 2988.146235] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2988.149269] ? apic_timer_interrupt+0xf/0x20 [ 2988.151639] [ 2988.152489] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 2988.155700] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 2988.158369] do_raw_spin_lock+0x177/0x190 [ 2988.160305] _raw_spin_lock+0x21/0x30 [ 2988.162294] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 2988.165207] obd_mod_free+0x46/0x140 [ptlrpc] [ 2988.167770] mdc_commit_open+0x91/0xb0 [mdc] [ 2988.169916] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 2988.172666] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 2988.175628] after_reply+0xe2c/0x1780 [ptlrpc] [ 2988.178311] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 2988.181614] ? __next_timer_interrupt+0x160/0x160 [ 2988.183676] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 2988.186557] ? do_wait_intr+0xf0/0xf0 [ 2988.188212] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 2988.191504] kthread+0x1d1/0x200 [ 2988.193779] ? set_kthread_struct+0x70/0x70 [ 2988.198753] ret_from_fork+0x1f/0x30 [ 2992.059248] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 2992.068156] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2992.099986] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2992.104200] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2992.107283] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2992.110711] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2992.117431] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 2992.120185] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2992.122828] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 2992.125505] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 2992.128045] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 2992.130457] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 2992.132751] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 2992.135450] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2992.137467] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 2992.139857] Call Trace: [ 2992.140714] [ 2992.141434] ? show_regs.cold.9+0x22/0x2f [ 2992.142914] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2992.144534] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2992.146383] ? watchdog+0x40/0x40 [ 2992.147993] ? hrtimer_interrupt+0x128/0x2c0 [ 2992.149643] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2992.152804] ? apic_timer_interrupt+0xf/0x20 [ 2992.156407] [ 2992.157546] ? native_safe_halt+0x2e/0x50 [ 2992.160188] kvm_wait+0xcd/0xe0 [ 2992.161474] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2992.163677] do_raw_spin_lock+0x177/0x190 [ 2992.165458] _raw_spin_lock+0x21/0x30 [ 2992.167135] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 2992.169992] ? do_raw_spin_unlock+0x75/0x190 [ 2992.171534] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 2992.173889] ? do_wait_intr+0xf0/0xf0 [ 2992.175235] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 2992.177594] mdc_reint+0x2d/0x110 [mdc] [ 2992.179256] mdc_unlink+0x658/0x870 [mdc] [ 2992.181102] lmv_unlink+0x34d/0xbd0 [lmv] [ 2992.182892] ll_unlink+0x24d/0x7f0 [lustre] [ 2992.184535] vfs_unlink+0x202/0x330 [ 2992.185992] do_unlinkat+0x2fa/0x460 [ 2992.187683] __x64_sys_unlink+0x37/0x50 [ 2992.189377] do_syscall_64+0xc1/0x440 [ 2992.190877] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 2992.192528] RIP: 0033:0x7f33957fc10b [ 2992.194150] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 2992.201143] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 2992.204376] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 2992.207088] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 2992.209674] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 2992.212429] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 2992.215374] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 2996.058033] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 2996.061248] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2996.082836] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 2996.088251] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 2996.092205] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 2996.095698] RIP: 0010:native_safe_halt+0x2e/0x50 [ 2996.097591] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 2996.105890] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 2996.109430] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 2996.112730] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 2996.115893] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 2996.118978] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 2996.122101] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 2996.125301] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 2996.129270] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2996.132091] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 2996.135029] Call Trace: [ 2996.136366] [ 2996.137394] ? show_regs.cold.9+0x22/0x2f [ 2996.139346] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 2996.141490] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 2996.143569] ? watchdog+0x40/0x40 [ 2996.145187] ? hrtimer_interrupt+0x128/0x2c0 [ 2996.147073] ? smp_apic_timer_interrupt+0xa6/0x270 [ 2996.149211] ? apic_timer_interrupt+0xf/0x20 [ 2996.151059] [ 2996.152080] ? native_safe_halt+0x2e/0x50 [ 2996.154119] kvm_wait+0xcd/0xe0 [ 2996.155472] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 2996.157846] do_raw_spin_lock+0x177/0x190 [ 2996.159918] _raw_spin_lock+0x21/0x30 [ 2996.161521] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 2996.164395] process_one_work+0x2c8/0x700 [ 2996.166304] worker_thread+0x296/0x6e0 [ 2996.167992] ? rescuer_thread+0x570/0x570 [ 2996.169734] kthread+0x1d1/0x200 [ 2996.171371] ? set_kthread_struct+0x70/0x70 [ 2996.173051] ret_from_fork+0x1f/0x30 [ 3016.053923] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3016.058124] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3016.085838] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3016.097593] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3016.104647] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3016.110589] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3016.135665] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3016.145203] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3016.150225] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3016.154172] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3016.158373] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3016.164268] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3016.167410] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3016.171660] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3016.174915] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3016.184501] Call Trace: [ 3016.186730] [ 3016.187639] ? show_regs.cold.9+0x22/0x2f [ 3016.194722] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3016.200744] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3016.205582] ? watchdog+0x40/0x40 [ 3016.209962] ? hrtimer_interrupt+0x128/0x2c0 [ 3016.215546] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3016.221057] ? apic_timer_interrupt+0xf/0x20 [ 3016.226683] [ 3016.228979] ? native_safe_halt+0x2e/0x50 [ 3016.233772] kvm_wait+0xcd/0xe0 [ 3016.236913] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3016.243553] do_raw_spin_lock+0x177/0x190 [ 3016.248059] _raw_spin_lock+0x21/0x30 [ 3016.253350] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3016.261904] obd_mod_free+0x46/0x140 [ptlrpc] [ 3016.271455] mdc_commit_open+0x91/0xb0 [mdc] [ 3016.277073] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3016.284905] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3016.293731] after_reply+0xe2c/0x1780 [ptlrpc] [ 3016.299338] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3016.307526] ? __next_timer_interrupt+0x160/0x160 [ 3016.313067] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3016.320088] ? do_wait_intr+0xf0/0xf0 [ 3016.324109] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3016.333095] kthread+0x1d1/0x200 [ 3016.336122] ? set_kthread_struct+0x70/0x70 [ 3016.348532] ret_from_fork+0x1f/0x30 [ 3020.059833] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3020.066036] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3020.115559] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3020.125491] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3020.137629] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3020.148355] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3020.170943] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3020.180604] RAX: 000000000000385d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3020.185979] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3020.192256] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3020.199424] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3020.203494] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3020.208536] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3020.214190] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3020.219042] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3020.231495] Call Trace: [ 3020.236709] [ 3020.245305] ? show_regs.cold.9+0x22/0x2f [ 3020.251292] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3020.258441] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3020.264944] ? watchdog+0x40/0x40 [ 3020.266913] ? hrtimer_interrupt+0x128/0x2c0 [ 3020.269940] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3020.273783] ? apic_timer_interrupt+0xf/0x20 [ 3020.276104] [ 3020.278265] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3020.281466] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3020.285418] do_raw_spin_lock+0x177/0x190 [ 3020.288008] _raw_spin_lock+0x21/0x30 [ 3020.290393] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3020.295899] ? do_raw_spin_unlock+0x75/0x190 [ 3020.299334] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3020.303636] ? do_wait_intr+0xf0/0xf0 [ 3020.308514] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3020.313909] mdc_reint+0x2d/0x110 [mdc] [ 3020.319030] mdc_unlink+0x658/0x870 [mdc] [ 3020.320616] lmv_unlink+0x34d/0xbd0 [lmv] [ 3020.323615] ll_unlink+0x24d/0x7f0 [lustre] [ 3020.326011] vfs_unlink+0x202/0x330 [ 3020.328598] do_unlinkat+0x2fa/0x460 [ 3020.330406] __x64_sys_unlink+0x37/0x50 [ 3020.334893] do_syscall_64+0xc1/0x440 [ 3020.337006] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3020.345557] RIP: 0033:0x7f33957fc10b [ 3020.350418] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3020.366425] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3020.387318] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3020.393783] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3020.403665] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3020.420887] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3020.426273] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3024.061120] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3024.065671] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3024.102905] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3024.110181] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3024.121120] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3024.129052] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3024.138093] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3024.152633] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 3024.161451] RAX: 0000000000003587 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3024.169110] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3024.186702] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3024.194404] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3024.199232] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3024.205835] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3024.218432] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3024.222215] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3024.230128] Call Trace: [ 3024.233996] [ 3024.237321] ? show_regs.cold.9+0x22/0x2f [ 3024.241891] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3024.247178] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3024.252059] ? watchdog+0x40/0x40 [ 3024.254089] ? hrtimer_interrupt+0x128/0x2c0 [ 3024.256586] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3024.264174] ? apic_timer_interrupt+0xf/0x20 [ 3024.267953] [ 3024.269074] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3024.275059] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3024.282914] do_raw_spin_lock+0x177/0x190 [ 3024.288771] _raw_spin_lock+0x21/0x30 [ 3024.292852] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3024.299825] process_one_work+0x2c8/0x700 [ 3024.305156] worker_thread+0x296/0x6e0 [ 3024.309878] ? rescuer_thread+0x570/0x570 [ 3024.313928] kthread+0x1d1/0x200 [ 3024.316410] ? set_kthread_struct+0x70/0x70 [ 3024.319855] ret_from_fork+0x1f/0x30 [ 3044.058925] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3044.073218] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3044.132733] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3044.149559] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3044.157282] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3044.161892] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3044.181819] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3044.190972] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3044.198195] RDX: 0000000000007f87 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3044.205795] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3044.218063] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3044.226962] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3044.231760] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3044.243687] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3044.250936] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3044.256106] Call Trace: [ 3044.257141] [ 3044.276155] ? show_regs.cold.9+0x22/0x2f [ 3044.277505] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3044.294129] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3044.303760] ? watchdog+0x40/0x40 [ 3044.307167] ? hrtimer_interrupt+0x128/0x2c0 [ 3044.315051] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3044.320145] ? apic_timer_interrupt+0xf/0x20 [ 3044.325800] [ 3044.327127] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3044.331596] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3044.341192] do_raw_spin_lock+0x177/0x190 [ 3044.344182] _raw_spin_lock+0x21/0x30 [ 3044.349102] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3044.353405] obd_mod_free+0x46/0x140 [ptlrpc] [ 3044.357967] mdc_commit_open+0x91/0xb0 [mdc] [ 3044.366757] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3044.371101] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3044.376847] after_reply+0xe2c/0x1780 [ptlrpc] [ 3044.380178] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3044.383841] ? __next_timer_interrupt+0x160/0x160 [ 3044.386579] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3044.389480] ? do_wait_intr+0xf0/0xf0 [ 3044.391214] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3044.393835] kthread+0x1d1/0x200 [ 3044.395378] ? set_kthread_struct+0x70/0x70 [ 3044.398034] ret_from_fork+0x1f/0x30 [ 3048.059280] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3048.076534] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3048.137092] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3048.154016] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3048.170175] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3048.176770] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3048.192697] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3048.197824] RAX: 0000000000005b0a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3048.205153] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3048.210778] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3048.217752] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3048.227512] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3048.233635] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3048.242778] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3048.248503] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3048.256893] Call Trace: [ 3048.257767] [ 3048.259292] ? show_regs.cold.9+0x22/0x2f [ 3048.264791] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3048.269049] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3048.271173] ? watchdog+0x40/0x40 [ 3048.272851] ? hrtimer_interrupt+0x128/0x2c0 [ 3048.277627] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3048.280124] ? apic_timer_interrupt+0xf/0x20 [ 3048.284473] [ 3048.285600] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3048.290164] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3048.292856] do_raw_spin_lock+0x177/0x190 [ 3048.298378] _raw_spin_lock+0x21/0x30 [ 3048.302766] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3048.306319] ? do_raw_spin_unlock+0x75/0x190 [ 3048.312469] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3048.319180] ? do_wait_intr+0xf0/0xf0 [ 3048.322656] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3048.329080] mdc_reint+0x2d/0x110 [mdc] [ 3048.333581] mdc_unlink+0x658/0x870 [mdc] [ 3048.340356] lmv_unlink+0x34d/0xbd0 [lmv] [ 3048.350651] ll_unlink+0x24d/0x7f0 [lustre] [ 3048.356591] vfs_unlink+0x202/0x330 [ 3048.367443] do_unlinkat+0x2fa/0x460 [ 3048.368805] __x64_sys_unlink+0x37/0x50 [ 3048.371878] do_syscall_64+0xc1/0x440 [ 3048.374502] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3048.383583] RIP: 0033:0x7f33957fc10b [ 3048.387065] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3048.406081] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3048.416480] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3048.420861] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3048.425848] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3048.433062] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3048.436367] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3052.058077] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3052.063060] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3052.089448] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3052.095046] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3052.099555] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3052.103222] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3052.106469] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3052.117177] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3052.121125] RAX: 0000000000001f11 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3052.125123] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3052.128911] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3052.132519] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3052.136493] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3052.140461] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3052.146054] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3052.149157] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3052.152710] Call Trace: [ 3052.154075] [ 3052.155330] ? show_regs.cold.9+0x22/0x2f [ 3052.157157] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3052.160305] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3052.163300] ? watchdog+0x40/0x40 [ 3052.165138] ? hrtimer_interrupt+0x128/0x2c0 [ 3052.166967] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3052.170027] ? apic_timer_interrupt+0xf/0x20 [ 3052.172477] [ 3052.174052] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3052.177024] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3052.181555] do_raw_spin_lock+0x177/0x190 [ 3052.185420] _raw_spin_lock+0x21/0x30 [ 3052.187357] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3052.193321] process_one_work+0x2c8/0x700 [ 3052.198161] worker_thread+0x296/0x6e0 [ 3052.200254] ? rescuer_thread+0x570/0x570 [ 3052.201725] kthread+0x1d1/0x200 [ 3052.203421] ? set_kthread_struct+0x70/0x70 [ 3052.205242] ret_from_fork+0x1f/0x30 [ 3058.042600] rcu: INFO: rcu_sched self-detected stall on CPU [ 3058.050625] rcu: 0-....: (575407 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=177607 [ 3058.059633] (t=960022 jiffies g=297885 q=6499) [ 3058.061957] NMI backtrace for cpu 0 [ 3058.063625] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3058.069270] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3058.073862] Call Trace: [ 3058.075537] [ 3058.076516] ? dump_stack+0xbb/0x10e [ 3058.078361] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3058.080618] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3058.083346] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3058.086283] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3058.088696] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3058.091305] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3058.093389] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3058.095935] ? update_process_times+0x6d/0xc0 [ 3058.098348] ? tick_sched_handle.isra.18+0x27/0x90 [ 3058.100407] ? tick_sched_timer+0x53/0xb0 [ 3058.103543] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3058.105970] ? tick_sched_handle.isra.18+0x90/0x90 [ 3058.108597] ? hrtimer_interrupt+0x128/0x2c0 [ 3058.111469] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3058.114929] ? apic_timer_interrupt+0xf/0x20 [ 3058.117969] [ 3058.119349] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3058.123767] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3058.127210] ? do_raw_spin_lock+0x177/0x190 [ 3058.129598] ? _raw_spin_lock+0x21/0x30 [ 3058.132067] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3058.136046] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3058.144816] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3058.148753] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3058.155196] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3058.159466] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3058.166939] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3058.171847] ? __next_timer_interrupt+0x160/0x160 [ 3058.174792] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3058.177836] ? do_wait_intr+0xf0/0xf0 [ 3058.180071] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3058.183156] ? kthread+0x1d1/0x200 [ 3058.185877] ? set_kthread_struct+0x70/0x70 [ 3058.188627] ? ret_from_fork+0x1f/0x30 [ 3076.059026] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3076.061724] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3076.087196] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3076.092419] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3076.096490] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3076.098382] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3076.120004] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3076.126814] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3076.135376] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3076.142845] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3076.147777] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3076.150961] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3076.155135] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3076.162952] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3076.167462] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3076.169978] Call Trace: [ 3076.171323] [ 3076.172074] ? show_regs.cold.9+0x22/0x2f [ 3076.174106] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3076.176343] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3076.180985] ? watchdog+0x40/0x40 [ 3076.183077] ? hrtimer_interrupt+0x128/0x2c0 [ 3076.184876] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3076.186568] ? apic_timer_interrupt+0xf/0x20 [ 3076.191282] [ 3076.193539] ? native_safe_halt+0x2e/0x50 [ 3076.196550] kvm_wait+0xcd/0xe0 [ 3076.197839] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3076.199979] do_raw_spin_lock+0x177/0x190 [ 3076.201388] _raw_spin_lock+0x21/0x30 [ 3076.203006] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3076.205136] ? do_raw_spin_unlock+0x75/0x190 [ 3076.209482] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3076.213406] ? do_wait_intr+0xf0/0xf0 [ 3076.214793] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3076.221156] mdc_reint+0x2d/0x110 [mdc] [ 3076.222603] mdc_unlink+0x658/0x870 [mdc] [ 3076.226986] lmv_unlink+0x34d/0xbd0 [lmv] [ 3076.228584] ll_unlink+0x24d/0x7f0 [lustre] [ 3076.231214] vfs_unlink+0x202/0x330 [ 3076.233294] do_unlinkat+0x2fa/0x460 [ 3076.234539] __x64_sys_unlink+0x37/0x50 [ 3076.236386] do_syscall_64+0xc1/0x440 [ 3076.237674] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3076.240987] RIP: 0033:0x7f33957fc10b [ 3076.249036] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3076.260438] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3076.264368] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3076.272133] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3076.280298] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3076.288813] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3076.293309] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3080.058322] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3080.062384] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3080.093095] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3080.107341] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3080.128632] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3080.135447] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3080.144284] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3080.176038] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3080.184680] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3080.197110] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3080.209503] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3080.218133] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3080.225288] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3080.234071] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3080.241698] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3080.253147] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3080.258160] Call Trace: [ 3080.260019] [ 3080.260802] ? show_regs.cold.9+0x22/0x2f [ 3080.264423] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3080.267463] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3080.276005] ? watchdog+0x40/0x40 [ 3080.279470] ? hrtimer_interrupt+0x128/0x2c0 [ 3080.285182] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3080.288438] ? apic_timer_interrupt+0xf/0x20 [ 3080.290144] [ 3080.292289] ? native_safe_halt+0x2e/0x50 [ 3080.293989] kvm_wait+0xcd/0xe0 [ 3080.296302] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3080.300092] do_raw_spin_lock+0x177/0x190 [ 3080.301459] _raw_spin_lock+0x21/0x30 [ 3080.303143] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3080.308048] process_one_work+0x2c8/0x700 [ 3080.310090] worker_thread+0x296/0x6e0 [ 3080.311381] ? rescuer_thread+0x570/0x570 [ 3080.315119] kthread+0x1d1/0x200 [ 3080.317463] ? set_kthread_struct+0x70/0x70 [ 3080.320499] ret_from_fork+0x1f/0x30 [ 3084.053040] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3084.056133] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3084.082433] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3084.087068] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3084.091226] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3084.093538] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3084.102364] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3084.105268] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3084.109542] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3084.113109] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3084.116159] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3084.118960] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3084.121734] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3084.125093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3084.127624] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3084.130388] Call Trace: [ 3084.131604] [ 3084.132293] ? show_regs.cold.9+0x22/0x2f [ 3084.138245] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3084.140269] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3084.142167] ? watchdog+0x40/0x40 [ 3084.143595] ? hrtimer_interrupt+0x128/0x2c0 [ 3084.145298] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3084.147168] ? apic_timer_interrupt+0xf/0x20 [ 3084.148975] [ 3084.149899] ? native_safe_halt+0x2e/0x50 [ 3084.151679] kvm_wait+0xcd/0xe0 [ 3084.152861] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3084.154464] do_raw_spin_lock+0x177/0x190 [ 3084.156374] _raw_spin_lock+0x21/0x30 [ 3084.158177] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3084.160348] obd_mod_free+0x46/0x140 [ptlrpc] [ 3084.162860] mdc_commit_open+0x91/0xb0 [mdc] [ 3084.165281] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3084.168081] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3084.170674] after_reply+0xe2c/0x1780 [ptlrpc] [ 3084.172834] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3084.175739] ? __next_timer_interrupt+0x160/0x160 [ 3084.178100] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3084.180386] ? do_wait_intr+0xf0/0xf0 [ 3084.182182] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3084.184703] kthread+0x1d1/0x200 [ 3084.186224] ? set_kthread_struct+0x70/0x70 [ 3084.188197] ret_from_fork+0x1f/0x30 [ 3104.059075] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3104.064993] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3104.100261] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3104.108175] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3104.114318] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3104.117203] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3104.130429] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3104.136056] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3104.147721] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3104.153498] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3104.160729] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3104.165474] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3104.171492] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3104.177433] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3104.182461] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3104.189611] Call Trace: [ 3104.191293] [ 3104.193276] ? show_regs.cold.9+0x22/0x2f [ 3104.197433] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3104.205245] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3104.209811] ? watchdog+0x40/0x40 [ 3104.212453] ? hrtimer_interrupt+0x128/0x2c0 [ 3104.217850] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3104.221886] ? apic_timer_interrupt+0xf/0x20 [ 3104.225493] [ 3104.226821] ? native_safe_halt+0x2e/0x50 [ 3104.230084] kvm_wait+0xcd/0xe0 [ 3104.232565] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3104.237308] do_raw_spin_lock+0x177/0x190 [ 3104.240480] _raw_spin_lock+0x21/0x30 [ 3104.243348] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3104.248156] ? do_raw_spin_unlock+0x75/0x190 [ 3104.252294] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3104.255793] ? do_wait_intr+0xf0/0xf0 [ 3104.258826] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3104.261817] mdc_reint+0x2d/0x110 [mdc] [ 3104.266568] mdc_unlink+0x658/0x870 [mdc] [ 3104.269522] lmv_unlink+0x34d/0xbd0 [lmv] [ 3104.273202] ll_unlink+0x24d/0x7f0 [lustre] [ 3104.277704] vfs_unlink+0x202/0x330 [ 3104.280527] do_unlinkat+0x2fa/0x460 [ 3104.283541] __x64_sys_unlink+0x37/0x50 [ 3104.287171] do_syscall_64+0xc1/0x440 [ 3104.289637] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3104.293064] RIP: 0033:0x7f33957fc10b [ 3104.296199] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3104.315316] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3104.321499] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3104.326658] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3104.332227] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3104.339704] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3104.346375] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3108.058089] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3108.061199] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3108.094860] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3108.104906] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3108.111164] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3108.115822] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3108.120302] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3108.135065] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3108.141602] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3108.148769] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3108.155206] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3108.161112] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3108.166529] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3108.185492] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3108.192196] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3108.200786] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3108.217534] Call Trace: [ 3108.220534] [ 3108.225550] ? show_regs.cold.9+0x22/0x2f [ 3108.231491] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3108.241088] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3108.247176] ? watchdog+0x40/0x40 [ 3108.252865] ? hrtimer_interrupt+0x128/0x2c0 [ 3108.259783] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3108.262984] ? apic_timer_interrupt+0xf/0x20 [ 3108.279796] [ 3108.282568] ? native_safe_halt+0x2e/0x50 [ 3108.286145] kvm_wait+0xcd/0xe0 [ 3108.288967] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3108.297083] do_raw_spin_lock+0x177/0x190 [ 3108.300315] _raw_spin_lock+0x21/0x30 [ 3108.303305] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3108.309562] process_one_work+0x2c8/0x700 [ 3108.317964] worker_thread+0x296/0x6e0 [ 3108.321513] ? rescuer_thread+0x570/0x570 [ 3108.324135] kthread+0x1d1/0x200 [ 3108.325484] ? set_kthread_struct+0x70/0x70 [ 3108.328833] ret_from_fork+0x1f/0x30 [ 3112.054060] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3112.064102] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3112.109426] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3112.117454] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3112.126065] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3112.129417] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3112.144459] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3112.151035] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3112.154879] RDX: 0000000000002dd5 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3112.159209] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3112.164611] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3112.169624] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3112.176536] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3112.181762] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3112.185897] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3112.189828] Call Trace: [ 3112.191838] [ 3112.193446] ? show_regs.cold.9+0x22/0x2f [ 3112.195006] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3112.198286] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3112.201563] ? watchdog+0x40/0x40 [ 3112.203165] ? hrtimer_interrupt+0x128/0x2c0 [ 3112.205908] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3112.209967] ? apic_timer_interrupt+0xf/0x20 [ 3112.211596] [ 3112.212374] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3112.216212] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3112.220453] do_raw_spin_lock+0x177/0x190 [ 3112.225670] _raw_spin_lock+0x21/0x30 [ 3112.229831] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3112.240246] obd_mod_free+0x46/0x140 [ptlrpc] [ 3112.244688] mdc_commit_open+0x91/0xb0 [mdc] [ 3112.250746] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3112.256597] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3112.261040] after_reply+0xe2c/0x1780 [ptlrpc] [ 3112.264505] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3112.271422] ? __next_timer_interrupt+0x160/0x160 [ 3112.274056] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3112.280554] ? do_wait_intr+0xf0/0xf0 [ 3112.283590] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3112.287949] kthread+0x1d1/0x200 [ 3112.290707] ? set_kthread_struct+0x70/0x70 [ 3112.293578] ret_from_fork+0x1f/0x30 [ 3132.060162] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3132.072035] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3132.124179] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3132.139394] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3132.149229] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3132.154048] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3132.165568] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 3132.173349] RAX: 0000000000001e4c RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3132.179358] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3132.185459] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3132.189959] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3132.194128] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3132.198514] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3132.203728] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3132.209012] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3132.212454] Call Trace: [ 3132.214994] [ 3132.217228] ? show_regs.cold.9+0x22/0x2f [ 3132.219981] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3132.223712] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3132.226095] ? watchdog+0x40/0x40 [ 3132.227980] ? hrtimer_interrupt+0x128/0x2c0 [ 3132.229981] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3132.232345] ? apic_timer_interrupt+0xf/0x20 [ 3132.233834] [ 3132.234705] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3132.237314] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3132.238880] do_raw_spin_lock+0x177/0x190 [ 3132.240413] _raw_spin_lock+0x21/0x30 [ 3132.241927] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3132.244293] ? do_raw_spin_unlock+0x75/0x190 [ 3132.245978] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3132.248277] ? do_wait_intr+0xf0/0xf0 [ 3132.249810] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3132.253319] mdc_reint+0x2d/0x110 [mdc] [ 3132.255982] mdc_unlink+0x658/0x870 [mdc] [ 3132.257424] lmv_unlink+0x34d/0xbd0 [lmv] [ 3132.265135] ll_unlink+0x24d/0x7f0 [lustre] [ 3132.266831] vfs_unlink+0x202/0x330 [ 3132.268894] do_unlinkat+0x2fa/0x460 [ 3132.270436] __x64_sys_unlink+0x37/0x50 [ 3132.271879] do_syscall_64+0xc1/0x440 [ 3132.273147] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3132.274947] RIP: 0033:0x7f33957fc10b [ 3132.276188] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3132.282752] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3132.287097] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3132.289510] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3132.291905] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3132.294423] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3132.296960] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3136.058102] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3136.068379] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3136.092407] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3136.097265] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3136.100904] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3136.104234] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3136.106249] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3136.114025] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3136.118199] RAX: 0000000000006b49 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3136.120818] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3136.124018] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3136.126870] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3136.130232] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3136.133297] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3136.137280] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3136.140099] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3136.144108] Call Trace: [ 3136.145250] [ 3136.146345] ? show_regs.cold.9+0x22/0x2f [ 3136.148067] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3136.149638] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3136.152291] ? watchdog+0x40/0x40 [ 3136.153714] ? hrtimer_interrupt+0x128/0x2c0 [ 3136.155834] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3136.158003] ? apic_timer_interrupt+0xf/0x20 [ 3136.161934] [ 3136.163772] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3136.168283] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3136.173122] do_raw_spin_lock+0x177/0x190 [ 3136.175964] _raw_spin_lock+0x21/0x30 [ 3136.180788] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3136.185891] process_one_work+0x2c8/0x700 [ 3136.187618] worker_thread+0x296/0x6e0 [ 3136.189985] ? rescuer_thread+0x570/0x570 [ 3136.195015] kthread+0x1d1/0x200 [ 3136.197745] ? set_kthread_struct+0x70/0x70 [ 3136.202024] ret_from_fork+0x1f/0x30 [ 3140.053105] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3140.058649] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3140.099100] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3140.105225] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3140.111748] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3140.114849] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3140.126026] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3140.133690] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3140.138059] RDX: 00000000000042ab RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3140.143389] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3140.148539] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3140.153720] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3140.157424] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3140.160640] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3140.164453] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3140.171304] Call Trace: [ 3140.176528] [ 3140.177753] ? show_regs.cold.9+0x22/0x2f [ 3140.182091] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3140.184646] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3140.186752] ? watchdog+0x40/0x40 [ 3140.188116] ? hrtimer_interrupt+0x128/0x2c0 [ 3140.190761] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3140.195682] ? apic_timer_interrupt+0xf/0x20 [ 3140.199710] [ 3140.200348] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3140.202895] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3140.206530] do_raw_spin_lock+0x177/0x190 [ 3140.211008] _raw_spin_lock+0x21/0x30 [ 3140.213775] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3140.217827] obd_mod_free+0x46/0x140 [ptlrpc] [ 3140.221119] mdc_commit_open+0x91/0xb0 [mdc] [ 3140.224019] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3140.227251] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3140.230515] after_reply+0xe2c/0x1780 [ptlrpc] [ 3140.234744] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3140.240987] ? __next_timer_interrupt+0x160/0x160 [ 3140.243691] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3140.249464] ? do_wait_intr+0xf0/0xf0 [ 3140.254251] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3140.257712] kthread+0x1d1/0x200 [ 3140.261177] ? set_kthread_struct+0x70/0x70 [ 3140.266302] ret_from_fork+0x1f/0x30 [ 3160.060361] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3160.063873] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3160.122136] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3160.132329] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3160.144197] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3160.152235] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3160.163302] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3160.167195] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3160.173438] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3160.180109] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3160.186790] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3160.194949] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3160.198548] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3160.202386] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3160.205719] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3160.212530] Call Trace: [ 3160.213622] [ 3160.214746] ? show_regs.cold.9+0x22/0x2f [ 3160.217451] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3160.221335] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3160.226043] ? watchdog+0x40/0x40 [ 3160.228497] ? hrtimer_interrupt+0x128/0x2c0 [ 3160.230765] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3160.235706] ? apic_timer_interrupt+0xf/0x20 [ 3160.239551] [ 3160.241885] ? native_safe_halt+0x2e/0x50 [ 3160.246276] kvm_wait+0xcd/0xe0 [ 3160.248289] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3160.251871] do_raw_spin_lock+0x177/0x190 [ 3160.259885] _raw_spin_lock+0x21/0x30 [ 3160.263227] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3160.266749] ? do_raw_spin_unlock+0x75/0x190 [ 3160.269906] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3160.273056] ? do_wait_intr+0xf0/0xf0 [ 3160.276122] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3160.279806] mdc_reint+0x2d/0x110 [mdc] [ 3160.283023] mdc_unlink+0x658/0x870 [mdc] [ 3160.285358] lmv_unlink+0x34d/0xbd0 [lmv] [ 3160.287769] ll_unlink+0x24d/0x7f0 [lustre] [ 3160.291455] vfs_unlink+0x202/0x330 [ 3160.293665] do_unlinkat+0x2fa/0x460 [ 3160.296125] __x64_sys_unlink+0x37/0x50 [ 3160.302309] do_syscall_64+0xc1/0x440 [ 3160.306206] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3160.309098] RIP: 0033:0x7f33957fc10b [ 3160.311166] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3160.328281] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3160.335655] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3160.341719] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3160.349543] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3160.355740] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3160.361647] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3164.058011] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3164.064846] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3164.107153] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3164.113436] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3164.120180] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3164.124184] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3164.132158] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3164.148632] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3164.151883] RAX: 00000000000062f8 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3164.155166] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3164.158227] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3164.160979] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3164.163816] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3164.166625] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3164.170084] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3164.172756] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3164.175197] Call Trace: [ 3164.176190] [ 3164.176969] ? show_regs.cold.9+0x22/0x2f [ 3164.178748] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3164.181540] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3164.183711] ? watchdog+0x40/0x40 [ 3164.185439] ? hrtimer_interrupt+0x128/0x2c0 [ 3164.187439] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3164.190128] ? apic_timer_interrupt+0xf/0x20 [ 3164.192023] [ 3164.192998] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3164.198439] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3164.201544] do_raw_spin_lock+0x177/0x190 [ 3164.203297] _raw_spin_lock+0x21/0x30 [ 3164.204578] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3164.207525] process_one_work+0x2c8/0x700 [ 3164.209103] worker_thread+0x296/0x6e0 [ 3164.210684] ? rescuer_thread+0x570/0x570 [ 3164.213520] kthread+0x1d1/0x200 [ 3164.214880] ? set_kthread_struct+0x70/0x70 [ 3164.216965] ret_from_fork+0x1f/0x30 [ 3168.053016] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3168.061766] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3168.087300] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3168.096564] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3168.100993] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3168.104629] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3168.117795] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3168.120455] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3168.126321] RDX: 00000000000064f2 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3168.129884] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3168.134074] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3168.136951] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3168.140341] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3168.143562] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3168.146051] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3168.148679] Call Trace: [ 3168.150769] [ 3168.152354] ? show_regs.cold.9+0x22/0x2f [ 3168.156460] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3168.158317] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3168.159879] ? watchdog+0x40/0x40 [ 3168.161994] ? hrtimer_interrupt+0x128/0x2c0 [ 3168.164329] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3168.166422] ? apic_timer_interrupt+0xf/0x20 [ 3168.172340] [ 3168.173353] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3168.176576] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3168.178735] do_raw_spin_lock+0x177/0x190 [ 3168.181308] _raw_spin_lock+0x21/0x30 [ 3168.186146] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3168.188989] obd_mod_free+0x46/0x140 [ptlrpc] [ 3168.193757] mdc_commit_open+0x91/0xb0 [mdc] [ 3168.196222] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3168.198736] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3168.203469] after_reply+0xe2c/0x1780 [ptlrpc] [ 3168.207018] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3168.210458] ? __next_timer_interrupt+0x160/0x160 [ 3168.212678] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3168.214627] ? do_wait_intr+0xf0/0xf0 [ 3168.215919] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3168.218573] kthread+0x1d1/0x200 [ 3168.220186] ? set_kthread_struct+0x70/0x70 [ 3168.223363] ret_from_fork+0x1f/0x30 [ 3188.059773] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3188.068071] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3188.105125] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3188.112299] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3188.119164] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3188.124708] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3188.140200] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3188.146485] RAX: 0000000000003883 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3188.152158] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3188.157276] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3188.164171] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3188.170697] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3188.176679] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3188.183181] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3188.187650] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3188.193911] Call Trace: [ 3188.197121] [ 3188.199557] ? show_regs.cold.9+0x22/0x2f [ 3188.202472] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3188.207276] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3188.211346] ? watchdog+0x40/0x40 [ 3188.216210] ? hrtimer_interrupt+0x128/0x2c0 [ 3188.218130] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3188.223822] ? apic_timer_interrupt+0xf/0x20 [ 3188.226210] [ 3188.230007] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3188.235510] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3188.238729] do_raw_spin_lock+0x177/0x190 [ 3188.244021] _raw_spin_lock+0x21/0x30 [ 3188.247158] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3188.253291] ? do_raw_spin_unlock+0x75/0x190 [ 3188.257076] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3188.262955] ? do_wait_intr+0xf0/0xf0 [ 3188.267773] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3188.270335] mdc_reint+0x2d/0x110 [mdc] [ 3188.275922] mdc_unlink+0x658/0x870 [mdc] [ 3188.279756] lmv_unlink+0x34d/0xbd0 [lmv] [ 3188.283236] ll_unlink+0x24d/0x7f0 [lustre] [ 3188.288776] vfs_unlink+0x202/0x330 [ 3188.291730] do_unlinkat+0x2fa/0x460 [ 3188.295933] __x64_sys_unlink+0x37/0x50 [ 3188.299020] do_syscall_64+0xc1/0x440 [ 3188.302911] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3188.306702] RIP: 0033:0x7f33957fc10b [ 3188.308606] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3188.325517] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3188.332530] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3188.339348] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3188.343307] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3188.353685] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3188.361533] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3192.058009] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3192.062322] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3192.095442] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3192.101261] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3192.110960] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3192.113037] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3192.117842] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3192.127200] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3192.133270] RAX: 0000000000000e6a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3192.139891] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3192.147280] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3192.151527] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3192.156060] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3192.159312] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3192.162706] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3192.165019] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3192.169836] Call Trace: [ 3192.171120] [ 3192.171946] ? show_regs.cold.9+0x22/0x2f [ 3192.173712] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3192.176023] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3192.178840] ? watchdog+0x40/0x40 [ 3192.181017] ? hrtimer_interrupt+0x128/0x2c0 [ 3192.183205] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3192.186172] ? apic_timer_interrupt+0xf/0x20 [ 3192.188534] [ 3192.189702] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3192.193610] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3192.196207] do_raw_spin_lock+0x177/0x190 [ 3192.198712] _raw_spin_lock+0x21/0x30 [ 3192.200703] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3192.203404] process_one_work+0x2c8/0x700 [ 3192.204375] worker_thread+0x296/0x6e0 [ 3192.207283] ? rescuer_thread+0x570/0x570 [ 3192.209822] kthread+0x1d1/0x200 [ 3192.211844] ? set_kthread_struct+0x70/0x70 [ 3192.214278] ret_from_fork+0x1f/0x30 [ 3196.055516] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3196.061769] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3196.103148] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3196.110602] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3196.118865] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3196.124751] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3196.139805] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3196.145970] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3196.151107] RDX: 00000000000020e7 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3196.157679] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3196.161861] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3196.170270] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3196.175159] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3196.189352] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3196.191479] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3196.201592] Call Trace: [ 3196.204918] [ 3196.206367] ? show_regs.cold.9+0x22/0x2f [ 3196.214934] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3196.216796] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3196.222902] ? watchdog+0x40/0x40 [ 3196.229359] ? hrtimer_interrupt+0x128/0x2c0 [ 3196.233694] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3196.236904] ? apic_timer_interrupt+0xf/0x20 [ 3196.243493] [ 3196.244752] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3196.251989] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3196.260031] do_raw_spin_lock+0x177/0x190 [ 3196.265777] _raw_spin_lock+0x21/0x30 [ 3196.268080] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3196.273068] obd_mod_free+0x46/0x140 [ptlrpc] [ 3196.277344] mdc_commit_open+0x91/0xb0 [mdc] [ 3196.280654] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3196.284784] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3196.288952] after_reply+0xe2c/0x1780 [ptlrpc] [ 3196.294139] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3196.302245] ? __next_timer_interrupt+0x160/0x160 [ 3196.306632] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3196.311443] ? do_wait_intr+0xf0/0xf0 [ 3196.316111] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3196.323096] kthread+0x1d1/0x200 [ 3196.329459] ? set_kthread_struct+0x70/0x70 [ 3196.333199] ret_from_fork+0x1f/0x30 [ 3216.059044] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3216.065276] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3216.091306] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3216.097985] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3216.101458] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3216.104330] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3216.113254] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3216.116378] RAX: 0000000000004f31 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3216.118401] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3216.121473] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3216.125028] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3216.127560] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3216.129328] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3216.132804] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3216.135481] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3216.140257] Call Trace: [ 3216.141503] [ 3216.142653] ? show_regs.cold.9+0x22/0x2f [ 3216.144930] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3216.147252] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3216.152916] ? watchdog+0x40/0x40 [ 3216.154506] ? hrtimer_interrupt+0x128/0x2c0 [ 3216.156780] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3216.159010] ? apic_timer_interrupt+0xf/0x20 [ 3216.160554] [ 3216.162119] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3216.165460] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3216.168131] do_raw_spin_lock+0x177/0x190 [ 3216.170980] _raw_spin_lock+0x21/0x30 [ 3216.172324] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3216.175161] ? do_raw_spin_unlock+0x75/0x190 [ 3216.177451] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3216.180261] ? do_wait_intr+0xf0/0xf0 [ 3216.182325] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3216.185952] mdc_reint+0x2d/0x110 [mdc] [ 3216.188117] mdc_unlink+0x658/0x870 [mdc] [ 3216.190432] lmv_unlink+0x34d/0xbd0 [lmv] [ 3216.192745] ll_unlink+0x24d/0x7f0 [lustre] [ 3216.195707] vfs_unlink+0x202/0x330 [ 3216.196994] do_unlinkat+0x2fa/0x460 [ 3216.198032] __x64_sys_unlink+0x37/0x50 [ 3216.200503] do_syscall_64+0xc1/0x440 [ 3216.202366] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3216.205001] RIP: 0033:0x7f33957fc10b [ 3216.210157] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3216.220096] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3216.223563] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3216.227138] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3216.230561] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3216.233741] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3216.236915] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3220.058096] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3220.066702] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3220.106703] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3220.122134] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3220.140298] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3220.158095] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3220.162885] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3220.188463] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3220.200195] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3220.212426] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3220.218326] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3220.222117] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3220.229156] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3220.245866] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3220.265473] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3220.275451] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3220.279414] Call Trace: [ 3220.282889] [ 3220.284300] ? show_regs.cold.9+0x22/0x2f [ 3220.286126] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3220.289068] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3220.293114] ? watchdog+0x40/0x40 [ 3220.295675] ? hrtimer_interrupt+0x128/0x2c0 [ 3220.302118] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3220.305737] ? apic_timer_interrupt+0xf/0x20 [ 3220.308990] [ 3220.310487] ? native_safe_halt+0x2e/0x50 [ 3220.313243] kvm_wait+0xcd/0xe0 [ 3220.315224] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3220.318740] do_raw_spin_lock+0x177/0x190 [ 3220.321200] _raw_spin_lock+0x21/0x30 [ 3220.323236] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3220.330096] process_one_work+0x2c8/0x700 [ 3220.334466] worker_thread+0x296/0x6e0 [ 3220.339527] ? rescuer_thread+0x570/0x570 [ 3220.343766] kthread+0x1d1/0x200 [ 3220.345841] ? set_kthread_struct+0x70/0x70 [ 3220.347365] ret_from_fork+0x1f/0x30 [ 3224.053007] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3224.057975] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3224.097224] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3224.110415] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3224.115563] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3224.119269] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3224.139306] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3224.143076] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3224.152156] RDX: 0000000000006a7f RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3224.158084] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3224.167151] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3224.171387] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3224.182318] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3224.188945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3224.192783] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3224.197228] Call Trace: [ 3224.199622] [ 3224.200969] ? show_regs.cold.9+0x22/0x2f [ 3224.203211] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3224.205923] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3224.208263] ? watchdog+0x40/0x40 [ 3224.209984] ? hrtimer_interrupt+0x128/0x2c0 [ 3224.212264] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3224.215364] ? apic_timer_interrupt+0xf/0x20 [ 3224.217530] [ 3224.218523] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3224.220910] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3224.223318] do_raw_spin_lock+0x177/0x190 [ 3224.225094] _raw_spin_lock+0x21/0x30 [ 3224.226581] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3224.228915] obd_mod_free+0x46/0x140 [ptlrpc] [ 3224.231117] mdc_commit_open+0x91/0xb0 [mdc] [ 3224.232868] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3224.237278] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3224.244351] after_reply+0xe2c/0x1780 [ptlrpc] [ 3224.247648] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3224.253032] ? __next_timer_interrupt+0x160/0x160 [ 3224.255937] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3224.257898] ? do_wait_intr+0xf0/0xf0 [ 3224.259028] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3224.263387] kthread+0x1d1/0x200 [ 3224.266835] ? set_kthread_struct+0x70/0x70 [ 3224.269439] ret_from_fork+0x1f/0x30 [ 3238.051327] rcu: INFO: rcu_sched self-detected stall on CPU [ 3238.060929] rcu: 0-....: (694366 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=212307 [ 3238.072794] (t=1140031 jiffies g=297885 q=7234) [ 3238.076519] NMI backtrace for cpu 0 [ 3238.079478] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3238.089225] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3238.096511] Call Trace: [ 3238.097984] [ 3238.099666] ? dump_stack+0xbb/0x10e [ 3238.102131] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3238.105707] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3238.108504] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3238.112099] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3238.115495] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3238.118683] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3238.122315] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3238.125589] ? update_process_times+0x6d/0xc0 [ 3238.128062] ? tick_sched_handle.isra.18+0x27/0x90 [ 3238.131469] ? tick_sched_timer+0x53/0xb0 [ 3238.133728] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3238.137565] ? tick_sched_handle.isra.18+0x90/0x90 [ 3238.140491] ? hrtimer_interrupt+0x128/0x2c0 [ 3238.142772] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3238.146182] ? apic_timer_interrupt+0xf/0x20 [ 3238.148708] [ 3238.150370] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3238.153639] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3238.158165] ? do_raw_spin_lock+0x177/0x190 [ 3238.162174] ? _raw_spin_lock+0x21/0x30 [ 3238.164482] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3238.167588] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3238.170732] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3238.173976] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3238.177295] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3238.180702] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3238.184075] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3238.187820] ? __next_timer_interrupt+0x160/0x160 [ 3238.190558] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3238.193329] ? do_wait_intr+0xf0/0xf0 [ 3238.195613] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3238.199306] ? kthread+0x1d1/0x200 [ 3238.201944] ? set_kthread_struct+0x70/0x70 [ 3238.204364] ? ret_from_fork+0x1f/0x30 [ 3244.059058] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3244.068935] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3244.096670] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3244.102099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3244.106852] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3244.110253] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3244.119602] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3244.123279] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3244.127135] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3244.130170] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3244.134728] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3244.138057] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3244.141142] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3244.145961] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3244.149281] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3244.152746] Call Trace: [ 3244.154244] [ 3244.156221] ? show_regs.cold.9+0x22/0x2f [ 3244.158355] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3244.160598] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3244.162269] ? watchdog+0x40/0x40 [ 3244.164372] ? hrtimer_interrupt+0x128/0x2c0 [ 3244.166643] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3244.168767] ? apic_timer_interrupt+0xf/0x20 [ 3244.170544] [ 3244.171637] ? native_safe_halt+0x2e/0x50 [ 3244.174246] kvm_wait+0xcd/0xe0 [ 3244.176181] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3244.178966] do_raw_spin_lock+0x177/0x190 [ 3244.181364] _raw_spin_lock+0x21/0x30 [ 3244.183246] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3244.185600] ? do_raw_spin_unlock+0x75/0x190 [ 3244.187214] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3244.189814] ? do_wait_intr+0xf0/0xf0 [ 3244.191179] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3244.193638] mdc_reint+0x2d/0x110 [mdc] [ 3244.195265] mdc_unlink+0x658/0x870 [mdc] [ 3244.197642] lmv_unlink+0x34d/0xbd0 [lmv] [ 3244.199231] ll_unlink+0x24d/0x7f0 [lustre] [ 3244.203006] vfs_unlink+0x202/0x330 [ 3244.204652] do_unlinkat+0x2fa/0x460 [ 3244.206212] __x64_sys_unlink+0x37/0x50 [ 3244.208088] do_syscall_64+0xc1/0x440 [ 3244.209902] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3244.212453] RIP: 0033:0x7f33957fc10b [ 3244.214368] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3244.223254] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3244.227380] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3244.230674] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3244.234171] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3244.238498] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3244.241938] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3248.060235] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3248.065577] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3248.102752] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3248.112035] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3248.119883] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3248.130085] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3248.134871] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3248.154758] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3248.161872] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3248.168386] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3248.180822] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3248.184426] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3248.187204] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3248.190113] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3248.193307] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3248.195837] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3248.198994] Call Trace: [ 3248.200060] [ 3248.200998] ? show_regs.cold.9+0x22/0x2f [ 3248.203025] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3248.205218] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3248.212481] ? watchdog+0x40/0x40 [ 3248.214660] ? hrtimer_interrupt+0x128/0x2c0 [ 3248.216922] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3248.219153] ? apic_timer_interrupt+0xf/0x20 [ 3248.224870] [ 3248.228555] ? native_safe_halt+0x2e/0x50 [ 3248.231201] kvm_wait+0xcd/0xe0 [ 3248.234465] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3248.236859] do_raw_spin_lock+0x177/0x190 [ 3248.238964] _raw_spin_lock+0x21/0x30 [ 3248.241240] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3248.248544] process_one_work+0x2c8/0x700 [ 3248.256749] worker_thread+0x296/0x6e0 [ 3248.259089] ? rescuer_thread+0x570/0x570 [ 3248.261694] kthread+0x1d1/0x200 [ 3248.264325] ? set_kthread_struct+0x70/0x70 [ 3248.266847] ret_from_fork+0x1f/0x30 [ 3264.053492] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3264.058167] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3264.105405] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3264.112963] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3264.119010] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3264.123706] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3264.140006] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3264.148704] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3264.152936] RDX: 0000000000004db1 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3264.156461] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3264.160883] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3264.165453] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3264.172080] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3264.177655] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3264.180837] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3264.185043] Call Trace: [ 3264.185891] [ 3264.187429] ? show_regs.cold.9+0x22/0x2f [ 3264.189299] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3264.191628] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3264.193865] ? watchdog+0x40/0x40 [ 3264.197795] ? hrtimer_interrupt+0x128/0x2c0 [ 3264.200940] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3264.205430] ? apic_timer_interrupt+0xf/0x20 [ 3264.210446] [ 3264.213128] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3264.218348] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3264.221981] do_raw_spin_lock+0x177/0x190 [ 3264.224705] _raw_spin_lock+0x21/0x30 [ 3264.228653] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3264.233082] obd_mod_free+0x46/0x140 [ptlrpc] [ 3264.236211] mdc_commit_open+0x91/0xb0 [mdc] [ 3264.238304] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3264.241468] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3264.244230] after_reply+0xe2c/0x1780 [ptlrpc] [ 3264.246888] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3264.249440] ? __next_timer_interrupt+0x160/0x160 [ 3264.252268] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3264.260957] ? do_wait_intr+0xf0/0xf0 [ 3264.263577] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3264.269713] kthread+0x1d1/0x200 [ 3264.274101] ? set_kthread_struct+0x70/0x70 [ 3264.276871] ret_from_fork+0x1f/0x30 [ 3272.059335] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3272.063129] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3272.085974] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3272.091286] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3272.096151] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3272.098481] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3272.113790] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 3272.121144] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3272.127283] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3272.132055] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3272.138934] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3272.143049] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3272.150992] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3272.156080] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3272.160245] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3272.163632] Call Trace: [ 3272.164828] [ 3272.165981] ? show_regs.cold.9+0x22/0x2f [ 3272.167371] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3272.170118] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3272.172392] ? watchdog+0x40/0x40 [ 3272.174447] ? hrtimer_interrupt+0x128/0x2c0 [ 3272.176979] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3272.179245] ? apic_timer_interrupt+0xf/0x20 [ 3272.181300] [ 3272.182261] ? native_safe_halt+0x2e/0x50 [ 3272.186109] kvm_wait+0xcd/0xe0 [ 3272.189192] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3272.192598] do_raw_spin_lock+0x177/0x190 [ 3272.197789] _raw_spin_lock+0x21/0x30 [ 3272.204838] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3272.214010] ? do_raw_spin_unlock+0x75/0x190 [ 3272.218370] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3272.226828] ? do_wait_intr+0xf0/0xf0 [ 3272.235501] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3272.240154] mdc_reint+0x2d/0x110 [mdc] [ 3272.242641] mdc_unlink+0x658/0x870 [mdc] [ 3272.246261] lmv_unlink+0x34d/0xbd0 [lmv] [ 3272.249195] ll_unlink+0x24d/0x7f0 [lustre] [ 3272.252294] vfs_unlink+0x202/0x330 [ 3272.254703] do_unlinkat+0x2fa/0x460 [ 3272.257470] __x64_sys_unlink+0x37/0x50 [ 3272.260112] do_syscall_64+0xc1/0x440 [ 3272.262227] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3272.264501] RIP: 0033:0x7f33957fc10b [ 3272.267206] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3272.280490] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3272.286248] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3272.290865] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3272.296464] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3272.306279] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3272.315188] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3276.060001] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 3276.068317] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3276.093331] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3276.099342] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3276.104817] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3276.113151] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3276.116342] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3276.126142] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3276.133914] RAX: 0000000000007751 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3276.137983] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3276.142236] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3276.145593] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3276.151534] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3276.157149] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3276.160924] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3276.163219] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3276.166209] Call Trace: [ 3276.167238] [ 3276.168047] ? show_regs.cold.9+0x22/0x2f [ 3276.169563] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3276.171222] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3276.172674] ? watchdog+0x40/0x40 [ 3276.173925] ? hrtimer_interrupt+0x128/0x2c0 [ 3276.175338] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3276.176914] ? apic_timer_interrupt+0xf/0x20 [ 3276.178252] [ 3276.179198] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3276.181049] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3276.182947] do_raw_spin_lock+0x177/0x190 [ 3276.184494] _raw_spin_lock+0x21/0x30 [ 3276.185999] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3276.189286] process_one_work+0x2c8/0x700 [ 3276.190821] worker_thread+0x296/0x6e0 [ 3276.192468] ? rescuer_thread+0x570/0x570 [ 3276.195782] kthread+0x1d1/0x200 [ 3276.196955] ? set_kthread_struct+0x70/0x70 [ 3276.199167] ret_from_fork+0x1f/0x30 [ 3292.053039] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3292.055615] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3292.080015] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3292.085289] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3292.090007] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3292.092275] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3292.100895] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3292.104559] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3292.110612] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3292.120093] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3292.129805] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3292.135998] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3292.144312] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3292.154165] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3292.160527] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3292.168169] Call Trace: [ 3292.171181] [ 3292.173415] ? show_regs.cold.9+0x22/0x2f [ 3292.179139] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3292.184631] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3292.188269] ? watchdog+0x40/0x40 [ 3292.193283] ? hrtimer_interrupt+0x128/0x2c0 [ 3292.198296] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3292.203235] ? apic_timer_interrupt+0xf/0x20 [ 3292.208474] [ 3292.209663] ? native_safe_halt+0x2e/0x50 [ 3292.212131] kvm_wait+0xcd/0xe0 [ 3292.214012] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3292.220145] do_raw_spin_lock+0x177/0x190 [ 3292.223422] _raw_spin_lock+0x21/0x30 [ 3292.229044] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3292.234796] obd_mod_free+0x46/0x140 [ptlrpc] [ 3292.246803] mdc_commit_open+0x91/0xb0 [mdc] [ 3292.254615] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3292.258484] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3292.266528] after_reply+0xe2c/0x1780 [ptlrpc] [ 3292.273055] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3292.278200] ? __next_timer_interrupt+0x160/0x160 [ 3292.280472] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3292.283432] ? do_wait_intr+0xf0/0xf0 [ 3292.285138] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3292.290146] kthread+0x1d1/0x200 [ 3292.292200] ? set_kthread_struct+0x70/0x70 [ 3292.294543] ret_from_fork+0x1f/0x30 [ 3300.061639] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3300.069320] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3300.110014] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3300.126705] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3300.141274] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3300.143959] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3300.158257] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3300.163339] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3300.166907] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3300.175718] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3300.188331] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3300.197742] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3300.202721] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3300.208668] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3300.217322] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3300.224590] Call Trace: [ 3300.229990] [ 3300.231689] ? show_regs.cold.9+0x22/0x2f [ 3300.238736] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3300.247151] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3300.248469] ? watchdog+0x40/0x40 [ 3300.255888] ? hrtimer_interrupt+0x128/0x2c0 [ 3300.262480] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3300.272166] ? apic_timer_interrupt+0xf/0x20 [ 3300.273711] [ 3300.281638] ? native_safe_halt+0x2e/0x50 [ 3300.286035] kvm_wait+0xcd/0xe0 [ 3300.288224] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3300.297203] do_raw_spin_lock+0x177/0x190 [ 3300.303025] _raw_spin_lock+0x21/0x30 [ 3300.305346] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3300.313367] ? do_raw_spin_unlock+0x75/0x190 [ 3300.316248] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3300.324405] ? do_wait_intr+0xf0/0xf0 [ 3300.329967] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3300.332119] mdc_reint+0x2d/0x110 [mdc] [ 3300.346665] mdc_unlink+0x658/0x870 [mdc] [ 3300.355793] lmv_unlink+0x34d/0xbd0 [lmv] [ 3300.359444] ll_unlink+0x24d/0x7f0 [lustre] [ 3300.365275] vfs_unlink+0x202/0x330 [ 3300.366699] do_unlinkat+0x2fa/0x460 [ 3300.369639] __x64_sys_unlink+0x37/0x50 [ 3300.373593] do_syscall_64+0xc1/0x440 [ 3300.381624] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3300.389706] RIP: 0033:0x7f33957fc10b [ 3300.393360] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3300.420340] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3300.424828] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3300.436022] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3300.439703] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3300.443043] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3300.445730] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3304.062138] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 3304.073536] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3304.160389] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3304.174664] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3304.189059] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3304.202359] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3304.214854] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3304.264638] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3304.281502] RAX: 0000000000007d80 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3304.295661] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3304.304450] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3304.314298] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3304.318425] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3304.330442] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3304.337555] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3304.340727] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3304.349136] Call Trace: [ 3304.350088] [ 3304.353745] ? show_regs.cold.9+0x22/0x2f [ 3304.359599] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3304.366779] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3304.372079] ? watchdog+0x40/0x40 [ 3304.375133] ? hrtimer_interrupt+0x128/0x2c0 [ 3304.376606] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3304.381828] ? apic_timer_interrupt+0xf/0x20 [ 3304.386489] [ 3304.389331] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3304.400785] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3304.406445] do_raw_spin_lock+0x177/0x190 [ 3304.417215] _raw_spin_lock+0x21/0x30 [ 3304.419675] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3304.427467] process_one_work+0x2c8/0x700 [ 3304.431362] worker_thread+0x296/0x6e0 [ 3304.436389] ? rescuer_thread+0x570/0x570 [ 3304.439708] kthread+0x1d1/0x200 [ 3304.446557] ? set_kthread_struct+0x70/0x70 [ 3304.453782] ret_from_fork+0x1f/0x30 [ 3320.055393] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3320.063360] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3320.110867] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3320.125973] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3320.135093] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3320.139648] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3320.170110] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3320.175615] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3320.183890] RDX: 0000000000007e7c RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3320.195984] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3320.206180] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3320.215413] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3320.221678] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3320.233131] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3320.238202] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3320.243383] Call Trace: [ 3320.244369] [ 3320.248748] ? show_regs.cold.9+0x22/0x2f [ 3320.255357] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3320.261303] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3320.266092] ? watchdog+0x40/0x40 [ 3320.267499] ? hrtimer_interrupt+0x128/0x2c0 [ 3320.269659] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3320.272498] ? apic_timer_interrupt+0xf/0x20 [ 3320.274992] [ 3320.275973] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3320.280189] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3320.283492] do_raw_spin_lock+0x177/0x190 [ 3320.286037] _raw_spin_lock+0x21/0x30 [ 3320.288145] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3320.293181] obd_mod_free+0x46/0x140 [ptlrpc] [ 3320.299330] mdc_commit_open+0x91/0xb0 [mdc] [ 3320.302520] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3320.309041] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3320.313474] after_reply+0xe2c/0x1780 [ptlrpc] [ 3320.317069] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3320.320882] ? __next_timer_interrupt+0x160/0x160 [ 3320.325993] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3320.330162] ? do_wait_intr+0xf0/0xf0 [ 3320.331409] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3320.337136] kthread+0x1d1/0x200 [ 3320.338701] ? set_kthread_struct+0x70/0x70 [ 3320.342748] ret_from_fork+0x1f/0x30 [ 3328.059177] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3328.066151] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3328.106466] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3328.115752] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3328.126865] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3328.133599] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3328.145506] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3328.149372] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3328.152495] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3328.155765] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3328.159161] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3328.162387] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3328.165851] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3328.170093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3328.173166] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3328.176347] Call Trace: [ 3328.177454] [ 3328.178531] ? show_regs.cold.9+0x22/0x2f [ 3328.180776] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3328.183769] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3328.185969] ? watchdog+0x40/0x40 [ 3328.187616] ? hrtimer_interrupt+0x128/0x2c0 [ 3328.189197] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3328.191228] ? apic_timer_interrupt+0xf/0x20 [ 3328.193350] [ 3328.194246] ? native_safe_halt+0x2e/0x50 [ 3328.196556] kvm_wait+0xcd/0xe0 [ 3328.199168] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3328.202783] do_raw_spin_lock+0x177/0x190 [ 3328.206596] _raw_spin_lock+0x21/0x30 [ 3328.208527] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3328.211820] ? do_raw_spin_unlock+0x75/0x190 [ 3328.213949] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3328.216531] ? do_wait_intr+0xf0/0xf0 [ 3328.218369] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3328.221378] mdc_reint+0x2d/0x110 [mdc] [ 3328.223189] mdc_unlink+0x658/0x870 [mdc] [ 3328.225225] lmv_unlink+0x34d/0xbd0 [lmv] [ 3328.227193] ll_unlink+0x24d/0x7f0 [lustre] [ 3328.229687] vfs_unlink+0x202/0x330 [ 3328.231590] do_unlinkat+0x2fa/0x460 [ 3328.233786] __x64_sys_unlink+0x37/0x50 [ 3328.235018] do_syscall_64+0xc1/0x440 [ 3328.236613] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3328.239091] RIP: 0033:0x7f33957fc10b [ 3328.240571] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3328.248849] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3328.252878] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3328.256952] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3328.264511] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3328.269434] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3328.277187] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3332.061228] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 3332.071780] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3332.108788] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3332.114993] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3332.122350] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3332.125751] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3332.130213] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3332.141862] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3332.145643] RAX: 000000000000630d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3332.153282] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3332.159415] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3332.166967] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3332.173027] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3332.177205] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3332.181199] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3332.185364] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3332.188441] Call Trace: [ 3332.189446] [ 3332.190413] ? show_regs.cold.9+0x22/0x2f [ 3332.192175] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3332.194451] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3332.199771] ? watchdog+0x40/0x40 [ 3332.201764] ? hrtimer_interrupt+0x128/0x2c0 [ 3332.203257] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3332.205323] ? apic_timer_interrupt+0xf/0x20 [ 3332.207021] [ 3332.207713] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3332.209733] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3332.211508] do_raw_spin_lock+0x177/0x190 [ 3332.212935] _raw_spin_lock+0x21/0x30 [ 3332.215124] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3332.222807] process_one_work+0x2c8/0x700 [ 3332.224536] worker_thread+0x296/0x6e0 [ 3332.228597] ? rescuer_thread+0x570/0x570 [ 3332.230483] kthread+0x1d1/0x200 [ 3332.233281] ? set_kthread_struct+0x70/0x70 [ 3332.235528] ret_from_fork+0x1f/0x30 [ 3348.055916] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3348.072392] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3348.134060] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3348.156648] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3348.166008] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3348.172041] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3348.197507] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3348.214958] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3348.221982] RDX: 000000000000148c RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3348.228684] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3348.246278] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3348.261609] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3348.273039] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3348.285633] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3348.293642] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3348.310380] Call Trace: [ 3348.311276] [ 3348.311956] ? show_regs.cold.9+0x22/0x2f [ 3348.319100] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3348.320566] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3348.325714] ? watchdog+0x40/0x40 [ 3348.326901] ? hrtimer_interrupt+0x128/0x2c0 [ 3348.330255] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3348.334551] ? apic_timer_interrupt+0xf/0x20 [ 3348.337715] [ 3348.339560] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3348.341457] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3348.344454] do_raw_spin_lock+0x177/0x190 [ 3348.347967] _raw_spin_lock+0x21/0x30 [ 3348.351345] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3348.353283] obd_mod_free+0x46/0x140 [ptlrpc] [ 3348.363392] mdc_commit_open+0x91/0xb0 [mdc] [ 3348.369380] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3348.373697] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3348.377411] after_reply+0xe2c/0x1780 [ptlrpc] [ 3348.383629] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3348.385706] ? __next_timer_interrupt+0x160/0x160 [ 3348.387361] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3348.396654] ? do_wait_intr+0xf0/0xf0 [ 3348.405849] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3348.412110] kthread+0x1d1/0x200 [ 3348.418691] ? set_kthread_struct+0x70/0x70 [ 3348.424287] ret_from_fork+0x1f/0x30 [ 3356.059059] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3356.067494] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3356.133211] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3356.148177] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3356.157249] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3356.161215] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3356.180387] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3356.187091] RAX: 0000000000001098 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3356.191520] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3356.195706] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3356.200437] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3356.206253] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3356.212222] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3356.219309] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3356.222872] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3356.232474] Call Trace: [ 3356.235202] [ 3356.238966] ? show_regs.cold.9+0x22/0x2f [ 3356.245390] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3356.251942] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3356.261432] ? watchdog+0x40/0x40 [ 3356.273979] ? hrtimer_interrupt+0x128/0x2c0 [ 3356.276242] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3356.286750] ? apic_timer_interrupt+0xf/0x20 [ 3356.289134] [ 3356.293755] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3356.298064] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3356.310273] do_raw_spin_lock+0x177/0x190 [ 3356.313533] _raw_spin_lock+0x21/0x30 [ 3356.316116] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3356.320566] ? do_raw_spin_unlock+0x75/0x190 [ 3356.324701] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3356.329572] ? do_wait_intr+0xf0/0xf0 [ 3356.333558] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3356.336087] mdc_reint+0x2d/0x110 [mdc] [ 3356.337869] mdc_unlink+0x658/0x870 [mdc] [ 3356.339980] lmv_unlink+0x34d/0xbd0 [lmv] [ 3356.342584] ll_unlink+0x24d/0x7f0 [lustre] [ 3356.344663] vfs_unlink+0x202/0x330 [ 3356.346286] do_unlinkat+0x2fa/0x460 [ 3356.347869] __x64_sys_unlink+0x37/0x50 [ 3356.349486] do_syscall_64+0xc1/0x440 [ 3356.350903] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3356.353120] RIP: 0033:0x7f33957fc10b [ 3356.357718] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3356.373091] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3356.378311] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3356.381886] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3356.389683] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3356.397100] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3356.403660] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3360.058028] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 3360.066727] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3360.131369] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3360.147158] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3360.154345] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3360.158581] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3360.162503] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3360.181374] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3360.191089] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3360.196359] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3360.204168] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3360.211996] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3360.216342] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3360.224770] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3360.236841] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3360.243628] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3360.252849] Call Trace: [ 3360.253792] [ 3360.256548] ? show_regs.cold.9+0x22/0x2f [ 3360.257821] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3360.269233] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3360.273273] ? watchdog+0x40/0x40 [ 3360.275413] ? hrtimer_interrupt+0x128/0x2c0 [ 3360.285905] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3360.289064] ? apic_timer_interrupt+0xf/0x20 [ 3360.293830] [ 3360.294478] ? native_safe_halt+0x2e/0x50 [ 3360.300468] kvm_wait+0xcd/0xe0 [ 3360.301702] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3360.303356] do_raw_spin_lock+0x177/0x190 [ 3360.305311] _raw_spin_lock+0x21/0x30 [ 3360.307166] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3360.316743] process_one_work+0x2c8/0x700 [ 3360.321040] worker_thread+0x296/0x6e0 [ 3360.326139] ? rescuer_thread+0x570/0x570 [ 3360.327467] kthread+0x1d1/0x200 [ 3360.329477] ? set_kthread_struct+0x70/0x70 [ 3360.332453] ret_from_fork+0x1f/0x30 [ 3376.053045] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3376.057702] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3376.080518] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3376.087258] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3376.093745] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3376.097182] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3376.108382] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3376.112075] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3376.116099] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3376.119624] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3376.123354] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3376.127517] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3376.131215] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3376.135999] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3376.140674] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3376.145481] Call Trace: [ 3376.146589] [ 3376.147801] ? show_regs.cold.9+0x22/0x2f [ 3376.152381] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3376.155009] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3376.157241] ? watchdog+0x40/0x40 [ 3376.161894] ? hrtimer_interrupt+0x128/0x2c0 [ 3376.165448] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3376.169016] ? apic_timer_interrupt+0xf/0x20 [ 3376.171636] [ 3376.172766] ? native_safe_halt+0x2e/0x50 [ 3376.175333] kvm_wait+0xcd/0xe0 [ 3376.177183] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3376.180170] do_raw_spin_lock+0x177/0x190 [ 3376.183173] _raw_spin_lock+0x21/0x30 [ 3376.189524] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3376.194082] obd_mod_free+0x46/0x140 [ptlrpc] [ 3376.201163] mdc_commit_open+0x91/0xb0 [mdc] [ 3376.207519] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3376.214153] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3376.221368] after_reply+0xe2c/0x1780 [ptlrpc] [ 3376.226195] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3376.231877] ? __next_timer_interrupt+0x160/0x160 [ 3376.237865] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3376.243399] ? do_wait_intr+0xf0/0xf0 [ 3376.246901] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3376.254007] kthread+0x1d1/0x200 [ 3376.255462] ? set_kthread_struct+0x70/0x70 [ 3376.262197] ret_from_fork+0x1f/0x30 [ 3384.063883] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3384.075632] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3384.114160] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3384.119377] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3384.127819] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3384.131991] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3384.140601] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3384.144316] RAX: 0000000000007cde RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3384.147788] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3384.153661] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3384.156899] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3384.159842] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3384.163748] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3384.170699] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3384.177267] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3384.183032] Call Trace: [ 3384.183940] [ 3384.186712] ? show_regs.cold.9+0x22/0x2f [ 3384.191221] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3384.193706] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3384.195619] ? watchdog+0x40/0x40 [ 3384.197594] ? hrtimer_interrupt+0x128/0x2c0 [ 3384.199715] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3384.204371] ? apic_timer_interrupt+0xf/0x20 [ 3384.208730] [ 3384.210155] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3384.222880] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3384.230749] do_raw_spin_lock+0x177/0x190 [ 3384.236413] _raw_spin_lock+0x21/0x30 [ 3384.239287] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3384.242521] ? do_raw_spin_unlock+0x75/0x190 [ 3384.244689] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3384.248231] ? do_wait_intr+0xf0/0xf0 [ 3384.254239] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3384.261760] mdc_reint+0x2d/0x110 [mdc] [ 3384.264691] mdc_unlink+0x658/0x870 [mdc] [ 3384.266484] lmv_unlink+0x34d/0xbd0 [lmv] [ 3384.268937] ll_unlink+0x24d/0x7f0 [lustre] [ 3384.272902] vfs_unlink+0x202/0x330 [ 3384.274347] do_unlinkat+0x2fa/0x460 [ 3384.276939] __x64_sys_unlink+0x37/0x50 [ 3384.279146] do_syscall_64+0xc1/0x440 [ 3384.280468] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3384.284314] RIP: 0033:0x7f33957fc10b [ 3384.285946] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3384.297579] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3384.301931] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3384.307252] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3384.311717] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3384.316154] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3384.323060] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3388.058028] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3388.068435] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3388.091372] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3388.096118] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3388.102952] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3388.107117] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3388.109290] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3388.124132] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3388.128925] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3388.133355] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3388.135817] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3388.139032] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3388.142388] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3388.146307] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3388.152346] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3388.155173] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3388.158656] Call Trace: [ 3388.160203] [ 3388.161267] ? show_regs.cold.9+0x22/0x2f [ 3388.163417] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3388.166702] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3388.169114] ? watchdog+0x40/0x40 [ 3388.170345] ? hrtimer_interrupt+0x128/0x2c0 [ 3388.171964] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3388.173924] ? apic_timer_interrupt+0xf/0x20 [ 3388.178082] [ 3388.179329] ? native_safe_halt+0x2e/0x50 [ 3388.184727] kvm_wait+0xcd/0xe0 [ 3388.187131] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3388.192183] do_raw_spin_lock+0x177/0x190 [ 3388.198025] _raw_spin_lock+0x21/0x30 [ 3388.201113] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3388.206216] process_one_work+0x2c8/0x700 [ 3388.208410] worker_thread+0x296/0x6e0 [ 3388.210004] ? rescuer_thread+0x570/0x570 [ 3388.212084] kthread+0x1d1/0x200 [ 3388.214570] ? set_kthread_struct+0x70/0x70 [ 3388.216862] ret_from_fork+0x1f/0x30 [ 3404.053096] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3404.059340] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3404.088256] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3404.094381] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3404.098317] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3404.100765] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3404.115790] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3404.120379] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3404.124435] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3404.128090] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3404.136484] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3404.142417] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3404.145657] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3404.149633] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3404.152204] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3404.155437] Call Trace: [ 3404.157208] [ 3404.158249] ? show_regs.cold.9+0x22/0x2f [ 3404.160301] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3404.163358] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3404.165213] ? watchdog+0x40/0x40 [ 3404.166492] ? hrtimer_interrupt+0x128/0x2c0 [ 3404.169308] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3404.173400] ? apic_timer_interrupt+0xf/0x20 [ 3404.175734] [ 3404.176880] ? native_safe_halt+0x2e/0x50 [ 3404.178988] kvm_wait+0xcd/0xe0 [ 3404.180147] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3404.187541] do_raw_spin_lock+0x177/0x190 [ 3404.189495] _raw_spin_lock+0x21/0x30 [ 3404.191106] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3404.195566] obd_mod_free+0x46/0x140 [ptlrpc] [ 3404.198354] mdc_commit_open+0x91/0xb0 [mdc] [ 3404.201237] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3404.204131] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3404.208511] after_reply+0xe2c/0x1780 [ptlrpc] [ 3404.214036] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3404.218307] ? __next_timer_interrupt+0x160/0x160 [ 3404.220917] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3404.224797] ? do_wait_intr+0xf0/0xf0 [ 3404.227868] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3404.231105] kthread+0x1d1/0x200 [ 3404.235872] ? set_kthread_struct+0x70/0x70 [ 3404.238421] ret_from_fork+0x1f/0x30 [ 3412.059100] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3412.062140] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3412.079577] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3412.082974] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3412.086254] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3412.088233] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3412.097537] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3412.103575] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3412.109667] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3412.112350] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3412.115479] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3412.118631] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3412.121843] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3412.130322] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3412.132638] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3412.136558] Call Trace: [ 3412.137377] [ 3412.138378] ? show_regs.cold.9+0x22/0x2f [ 3412.140351] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3412.142165] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3412.143694] ? watchdog+0x40/0x40 [ 3412.144717] ? hrtimer_interrupt+0x128/0x2c0 [ 3412.146771] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3412.148635] ? apic_timer_interrupt+0xf/0x20 [ 3412.150441] [ 3412.151318] ? native_safe_halt+0x2e/0x50 [ 3412.153152] kvm_wait+0xcd/0xe0 [ 3412.154246] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3412.156295] do_raw_spin_lock+0x177/0x190 [ 3412.157432] _raw_spin_lock+0x21/0x30 [ 3412.159210] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3412.161827] ? do_raw_spin_unlock+0x75/0x190 [ 3412.164971] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3412.168277] ? do_wait_intr+0xf0/0xf0 [ 3412.169519] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3412.171757] mdc_reint+0x2d/0x110 [mdc] [ 3412.173852] mdc_unlink+0x658/0x870 [mdc] [ 3412.175775] lmv_unlink+0x34d/0xbd0 [lmv] [ 3412.179167] ll_unlink+0x24d/0x7f0 [lustre] [ 3412.180819] vfs_unlink+0x202/0x330 [ 3412.182331] do_unlinkat+0x2fa/0x460 [ 3412.185707] __x64_sys_unlink+0x37/0x50 [ 3412.188391] do_syscall_64+0xc1/0x440 [ 3412.191368] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3412.193901] RIP: 0033:0x7f33957fc10b [ 3412.196077] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3412.206601] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3412.212950] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3412.218161] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3412.221426] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3412.229964] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3412.236889] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3416.058080] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3416.063403] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3416.094417] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3416.109593] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3416.118713] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3416.126793] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3416.131832] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3416.151804] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3416.156061] RAX: 0000000000003329 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3416.162936] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3416.167263] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3416.171565] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3416.175494] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3416.178763] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3416.183115] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3416.185682] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3416.188621] Call Trace: [ 3416.189634] [ 3416.190571] ? show_regs.cold.9+0x22/0x2f [ 3416.192624] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3416.195608] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3416.197611] ? watchdog+0x40/0x40 [ 3416.199186] ? hrtimer_interrupt+0x128/0x2c0 [ 3416.201442] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3416.206084] ? apic_timer_interrupt+0xf/0x20 [ 3416.208738] [ 3416.211082] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3416.215016] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3416.218742] do_raw_spin_lock+0x177/0x190 [ 3416.220503] _raw_spin_lock+0x21/0x30 [ 3416.222358] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3416.226096] process_one_work+0x2c8/0x700 [ 3416.227417] worker_thread+0x296/0x6e0 [ 3416.233692] ? rescuer_thread+0x570/0x570 [ 3416.235736] kthread+0x1d1/0x200 [ 3416.237699] ? set_kthread_struct+0x70/0x70 [ 3416.243140] ret_from_fork+0x1f/0x30 [ 3418.054102] rcu: INFO: rcu_sched self-detected stall on CPU [ 3418.058709] rcu: 0-....: (814468 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=247154 [ 3418.066971] (t=1320034 jiffies g=297885 q=7925) [ 3418.071624] NMI backtrace for cpu 0 [ 3418.078427] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3418.093924] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3418.101508] Call Trace: [ 3418.103795] [ 3418.105110] ? dump_stack+0xbb/0x10e [ 3418.109159] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3418.111794] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3418.113792] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3418.120297] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3418.123391] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3418.128671] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3418.134387] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3418.137718] ? update_process_times+0x6d/0xc0 [ 3418.142447] ? tick_sched_handle.isra.18+0x27/0x90 [ 3418.150812] ? tick_sched_timer+0x53/0xb0 [ 3418.157336] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3418.164192] ? tick_sched_handle.isra.18+0x90/0x90 [ 3418.170492] ? hrtimer_interrupt+0x128/0x2c0 [ 3418.174493] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3418.177596] ? apic_timer_interrupt+0xf/0x20 [ 3418.184397] [ 3418.186062] ? native_safe_halt+0x2e/0x50 [ 3418.188593] ? kvm_wait+0xcd/0xe0 [ 3418.191046] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3418.198001] ? do_raw_spin_lock+0x177/0x190 [ 3418.201984] ? _raw_spin_lock+0x21/0x30 [ 3418.205625] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3418.209663] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3418.214369] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3418.217646] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3418.221040] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3418.224736] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3418.229086] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3418.232114] ? __next_timer_interrupt+0x160/0x160 [ 3418.234693] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3418.237485] ? do_wait_intr+0xf0/0xf0 [ 3418.239578] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3418.242898] ? kthread+0x1d1/0x200 [ 3418.245106] ? set_kthread_struct+0x70/0x70 [ 3418.247504] ? ret_from_fork+0x1f/0x30 [ 3440.061522] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3440.071115] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3440.149055] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3440.166899] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3440.177939] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3440.185253] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3440.208699] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3440.220156] RAX: 0000000000007185 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3440.225123] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3440.231366] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3440.235103] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3440.240738] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3440.243217] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3440.247812] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3440.250344] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3440.262622] Call Trace: [ 3440.263458] [ 3440.267350] ? show_regs.cold.9+0x22/0x2f [ 3440.272042] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3440.278456] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3440.279914] ? watchdog+0x40/0x40 [ 3440.283259] ? hrtimer_interrupt+0x128/0x2c0 [ 3440.288110] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3440.290319] ? apic_timer_interrupt+0xf/0x20 [ 3440.292284] [ 3440.293233] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3440.295640] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3440.297633] do_raw_spin_lock+0x177/0x190 [ 3440.304124] _raw_spin_lock+0x21/0x30 [ 3440.307140] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3440.310298] ? do_raw_spin_unlock+0x75/0x190 [ 3440.314258] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3440.317936] ? do_wait_intr+0xf0/0xf0 [ 3440.319569] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3440.322392] mdc_reint+0x2d/0x110 [mdc] [ 3440.324470] mdc_unlink+0x658/0x870 [mdc] [ 3440.327844] lmv_unlink+0x34d/0xbd0 [lmv] [ 3440.329283] ll_unlink+0x24d/0x7f0 [lustre] [ 3440.332849] vfs_unlink+0x202/0x330 [ 3440.334513] do_unlinkat+0x2fa/0x460 [ 3440.336283] __x64_sys_unlink+0x37/0x50 [ 3440.339383] do_syscall_64+0xc1/0x440 [ 3440.341854] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3440.345109] RIP: 0033:0x7f33957fc10b [ 3440.347539] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3440.358386] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3440.361958] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3440.365384] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3440.371230] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3440.377761] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3440.382272] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3444.053028] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3444.057800] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3444.058129] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3444.082587] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3444.082600] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3444.082604] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3444.082619] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3444.082623] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3444.082629] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3444.082632] RDX: 0000000000002a1e RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3444.082635] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3444.082639] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3444.085555] Modules linked in: [ 3444.090477] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3444.094119] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3444.094210] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3444.094224] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3444.094231] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3444.094827] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3444.094838] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3444.094842] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3444.094847] RAX: 0000000000007b11 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3444.094850] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3444.094852] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3444.094854] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3444.094857] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3444.094860] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3444.094866] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3444.094869] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3444.094876] Call Trace: [ 3444.094882] [ 3444.094887] ? show_regs.cold.9+0x22/0x2f [ 3444.094901] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3444.094908] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3444.094912] ? watchdog+0x40/0x40 [ 3444.094917] ? hrtimer_interrupt+0x128/0x2c0 [ 3444.094921] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3444.094930] ? apic_timer_interrupt+0xf/0x20 [ 3444.094934] [ 3444.094937] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3444.094942] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3444.094947] do_raw_spin_lock+0x177/0x190 [ 3444.095104] _raw_spin_lock+0x21/0x30 [ 3444.095109] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3444.095457] process_one_work+0x2c8/0x700 [ 3444.095466] worker_thread+0x296/0x6e0 [ 3444.095473] ? rescuer_thread+0x570/0x570 [ 3444.095477] kthread+0x1d1/0x200 [ 3444.095482] ? set_kthread_struct+0x70/0x70 [ 3444.095485] ret_from_fork+0x1f/0x30 [ 3444.328841] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3444.339697] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3444.343295] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3444.354178] Call Trace: [ 3444.356870] [ 3444.358779] ? show_regs.cold.9+0x22/0x2f [ 3444.360855] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3444.364216] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3444.367044] ? watchdog+0x40/0x40 [ 3444.368747] ? hrtimer_interrupt+0x128/0x2c0 [ 3444.371129] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3444.375226] ? apic_timer_interrupt+0xf/0x20 [ 3444.377312] [ 3444.378178] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3444.381044] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3444.383785] do_raw_spin_lock+0x177/0x190 [ 3444.385739] _raw_spin_lock+0x21/0x30 [ 3444.387887] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3444.390227] obd_mod_free+0x46/0x140 [ptlrpc] [ 3444.393602] mdc_commit_open+0x91/0xb0 [mdc] [ 3444.398521] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3444.404851] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3444.412505] after_reply+0xe2c/0x1780 [ptlrpc] [ 3444.417825] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3444.423365] ? __next_timer_interrupt+0x160/0x160 [ 3444.426446] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3444.429566] ? do_wait_intr+0xf0/0xf0 [ 3444.435868] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3444.441879] kthread+0x1d1/0x200 [ 3444.448592] ? set_kthread_struct+0x70/0x70 [ 3444.456688] ret_from_fork+0x1f/0x30 [ 3468.059037] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3468.065006] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3468.102803] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3468.111552] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3468.119679] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3468.124006] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3468.140423] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3468.144171] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3468.150233] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3468.156860] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3468.162521] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3468.166827] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3468.176859] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3468.187019] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3468.192657] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3468.196562] Call Trace: [ 3468.197772] [ 3468.199685] ? show_regs.cold.9+0x22/0x2f [ 3468.205623] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3468.212043] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3468.217492] ? watchdog+0x40/0x40 [ 3468.220375] ? hrtimer_interrupt+0x128/0x2c0 [ 3468.225877] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3468.230043] ? apic_timer_interrupt+0xf/0x20 [ 3468.237331] [ 3468.239442] ? native_safe_halt+0x2e/0x50 [ 3468.244539] kvm_wait+0xcd/0xe0 [ 3468.246437] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3468.248474] do_raw_spin_lock+0x177/0x190 [ 3468.252779] _raw_spin_lock+0x21/0x30 [ 3468.255647] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3468.261048] ? do_raw_spin_unlock+0x75/0x190 [ 3468.266137] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3468.274418] ? do_wait_intr+0xf0/0xf0 [ 3468.277212] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3468.280911] mdc_reint+0x2d/0x110 [mdc] [ 3468.284448] mdc_unlink+0x658/0x870 [mdc] [ 3468.289095] lmv_unlink+0x34d/0xbd0 [lmv] [ 3468.300875] ll_unlink+0x24d/0x7f0 [lustre] [ 3468.306410] vfs_unlink+0x202/0x330 [ 3468.308828] do_unlinkat+0x2fa/0x460 [ 3468.312078] __x64_sys_unlink+0x37/0x50 [ 3468.316103] do_syscall_64+0xc1/0x440 [ 3468.317794] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3468.327650] RIP: 0033:0x7f33957fc10b [ 3468.331745] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3468.359366] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3468.367412] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3468.373510] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3468.379550] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3468.386510] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3468.392449] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3472.053072] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3472.057409] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3472.059166] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3472.080658] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3472.080672] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3472.080675] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3472.080687] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3472.080696] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3472.080702] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3472.080704] RDX: 00000000000008b5 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3472.080706] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3472.080708] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3472.080710] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3472.080712] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3472.080714] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3472.080716] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3472.080723] Call Trace: [ 3472.080728] [ 3472.080733] ? show_regs.cold.9+0x22/0x2f [ 3472.080741] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3472.080746] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3472.080750] ? watchdog+0x40/0x40 [ 3472.080754] ? hrtimer_interrupt+0x128/0x2c0 [ 3472.080757] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3472.080766] ? apic_timer_interrupt+0xf/0x20 [ 3472.080770] [ 3472.080832] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3472.084363] Modules linked in: [ 3472.089223] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3472.093023] lustre(O) osp(O) [ 3472.096152] do_raw_spin_lock+0x177/0x190 [ 3472.105091] ofd(O) [ 3472.108577] _raw_spin_lock+0x21/0x30 [ 3472.108601] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3472.115001] lod(O) [ 3472.118188] obd_mod_free+0x46/0x140 [ptlrpc] [ 3472.118524] mdc_commit_open+0x91/0xb0 [mdc] [ 3472.118560] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3472.118911] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3472.119130] after_reply+0xe2c/0x1780 [ptlrpc] [ 3472.119369] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3472.119598] ? __next_timer_interrupt+0x160/0x160 [ 3472.119681] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3472.119956] ? do_wait_intr+0xf0/0xf0 [ 3472.119966] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3472.120200] kthread+0x1d1/0x200 [ 3472.120291] ? set_kthread_struct+0x70/0x70 [ 3472.120295] ret_from_fork+0x1f/0x30 [ 3472.214963] mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3472.233478] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3472.238408] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3472.242403] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3472.245343] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3472.248976] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3472.259293] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3472.263967] RAX: 0000000000007e93 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3472.268141] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3472.271314] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3472.274363] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3472.279382] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3472.284866] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3472.288309] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3472.290823] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3472.293830] Call Trace: [ 3472.294990] [ 3472.296241] ? show_regs.cold.9+0x22/0x2f [ 3472.298249] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3472.300328] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3472.303303] ? watchdog+0x40/0x40 [ 3472.305389] ? hrtimer_interrupt+0x128/0x2c0 [ 3472.308744] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3472.311720] ? apic_timer_interrupt+0xf/0x20 [ 3472.313760] [ 3472.314847] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3472.319268] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3472.326318] do_raw_spin_lock+0x177/0x190 [ 3472.328440] _raw_spin_lock+0x21/0x30 [ 3472.329712] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3472.334439] process_one_work+0x2c8/0x700 [ 3472.336216] worker_thread+0x296/0x6e0 [ 3472.338707] ? rescuer_thread+0x570/0x570 [ 3472.340558] kthread+0x1d1/0x200 [ 3472.341886] ? set_kthread_struct+0x70/0x70 [ 3472.343347] ret_from_fork+0x1f/0x30 [ 3496.059189] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3496.062031] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3496.092864] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3496.099055] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3496.102717] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3496.104351] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3496.115739] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3496.118589] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3496.124962] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3496.127145] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3496.130287] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3496.136809] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3496.143624] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3496.151882] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3496.157087] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3496.160929] Call Trace: [ 3496.162020] [ 3496.163176] ? show_regs.cold.9+0x22/0x2f [ 3496.165366] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3496.167460] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3496.169201] ? watchdog+0x40/0x40 [ 3496.172636] ? hrtimer_interrupt+0x128/0x2c0 [ 3496.176592] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3496.183985] ? apic_timer_interrupt+0xf/0x20 [ 3496.187761] [ 3496.189857] ? native_safe_halt+0x2e/0x50 [ 3496.193107] kvm_wait+0xcd/0xe0 [ 3496.197060] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3496.202859] do_raw_spin_lock+0x177/0x190 [ 3496.204542] _raw_spin_lock+0x21/0x30 [ 3496.209456] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3496.214481] ? do_raw_spin_unlock+0x75/0x190 [ 3496.219201] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3496.225875] ? do_wait_intr+0xf0/0xf0 [ 3496.229223] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3496.235879] mdc_reint+0x2d/0x110 [mdc] [ 3496.238936] mdc_unlink+0x658/0x870 [mdc] [ 3496.243491] lmv_unlink+0x34d/0xbd0 [lmv] [ 3496.247252] ll_unlink+0x24d/0x7f0 [lustre] [ 3496.250865] vfs_unlink+0x202/0x330 [ 3496.254442] do_unlinkat+0x2fa/0x460 [ 3496.256169] __x64_sys_unlink+0x37/0x50 [ 3496.260759] do_syscall_64+0xc1/0x440 [ 3496.263172] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3496.269776] RIP: 0033:0x7f33957fc10b [ 3496.273671] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3496.297200] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3496.305821] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3496.312767] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3496.317154] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3496.320351] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3496.325800] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3500.053197] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3500.058031] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3500.058582] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3500.089969] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3500.089983] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3500.089987] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3500.094562] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3500.094643] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3500.094661] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3500.094667] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3500.099946] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3500.099959] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3500.099966] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3500.099969] RDX: 00000000000060c3 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3500.099971] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3500.099973] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3500.099975] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3500.099978] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3500.099981] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3500.099984] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3500.099992] Call Trace: [ 3500.099997] [ 3500.100003] ? show_regs.cold.9+0x22/0x2f [ 3500.100018] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3500.100025] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3500.100030] ? watchdog+0x40/0x40 [ 3500.100035] ? hrtimer_interrupt+0x128/0x2c0 [ 3500.100039] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3500.100048] ? apic_timer_interrupt+0xf/0x20 [ 3500.100053] [ 3500.100056] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3500.100063] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3500.100068] do_raw_spin_lock+0x177/0x190 [ 3500.100073] _raw_spin_lock+0x21/0x30 [ 3500.100079] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3500.110017] [ 3500.110029] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3500.110048] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3500.110053] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3500.110058] RAX: 000000000000650f RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3500.110060] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3500.110062] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3500.110064] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3500.110067] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3500.110069] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3500.110072] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3500.110074] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3500.110084] Call Trace: [ 3500.110090] [ 3500.110096] ? show_regs.cold.9+0x22/0x2f [ 3500.110105] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3500.110112] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3500.110122] ? watchdog+0x40/0x40 [ 3500.112631] obd_mod_free+0x46/0x140 [ptlrpc] [ 3500.113185] mdc_commit_open+0x91/0xb0 [mdc] [ 3500.143771] ? hrtimer_interrupt+0x128/0x2c0 [ 3500.150570] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3500.152081] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3500.158553] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3500.158575] ? apic_timer_interrupt+0xf/0x20 [ 3500.158580] [ 3500.158584] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3500.158591] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3500.158597] do_raw_spin_lock+0x177/0x190 [ 3500.158603] _raw_spin_lock+0x21/0x30 [ 3500.158608] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3500.158896] process_one_work+0x2c8/0x700 [ 3500.158902] worker_thread+0x296/0x6e0 [ 3500.158906] ? rescuer_thread+0x570/0x570 [ 3500.158909] kthread+0x1d1/0x200 [ 3500.158912] ? set_kthread_struct+0x70/0x70 [ 3500.158916] ret_from_fork+0x1f/0x30 [ 3500.345841] after_reply+0xe2c/0x1780 [ptlrpc] [ 3500.347978] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3500.350140] ? __next_timer_interrupt+0x160/0x160 [ 3500.351793] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3500.353583] ? do_wait_intr+0xf0/0xf0 [ 3500.354819] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3500.356866] kthread+0x1d1/0x200 [ 3500.358064] ? set_kthread_struct+0x70/0x70 [ 3500.359530] ret_from_fork+0x1f/0x30 [ 3524.061780] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3524.070218] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3524.118236] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3524.128443] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3524.137323] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3524.143644] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3524.158645] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3524.161853] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3524.169652] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3524.181629] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3524.188881] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3524.193879] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3524.198234] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3524.206378] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3524.210163] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3524.213540] Call Trace: [ 3524.217607] [ 3524.218856] ? show_regs.cold.9+0x22/0x2f [ 3524.220734] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3524.225082] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3524.230888] ? watchdog+0x40/0x40 [ 3524.232403] ? hrtimer_interrupt+0x128/0x2c0 [ 3524.237186] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3524.240430] ? apic_timer_interrupt+0xf/0x20 [ 3524.245954] [ 3524.247494] ? native_safe_halt+0x2e/0x50 [ 3524.252769] kvm_wait+0xcd/0xe0 [ 3524.253639] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3524.263192] do_raw_spin_lock+0x177/0x190 [ 3524.268478] _raw_spin_lock+0x21/0x30 [ 3524.274492] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3524.279402] ? do_raw_spin_unlock+0x75/0x190 [ 3524.287221] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3524.292116] ? do_wait_intr+0xf0/0xf0 [ 3524.294166] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3524.298729] mdc_reint+0x2d/0x110 [mdc] [ 3524.302879] mdc_unlink+0x658/0x870 [mdc] [ 3524.306226] lmv_unlink+0x34d/0xbd0 [lmv] [ 3524.312729] ll_unlink+0x24d/0x7f0 [lustre] [ 3524.314194] vfs_unlink+0x202/0x330 [ 3524.320646] do_unlinkat+0x2fa/0x460 [ 3524.325241] __x64_sys_unlink+0x37/0x50 [ 3524.328790] do_syscall_64+0xc1/0x440 [ 3524.330540] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3524.341103] RIP: 0033:0x7f33957fc10b [ 3524.347131] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3524.377689] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3524.386551] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3524.396133] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3524.408405] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3524.414255] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3524.419286] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3528.058177] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3528.061603] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3528.061828] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3528.082948] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3528.087243] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3528.087322] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3528.087329] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3528.087336] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3528.087630] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3528.087831] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3528.087836] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3528.087840] RAX: 0000000000004f60 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3528.087843] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3528.087845] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3528.087847] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3528.087849] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3528.087852] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3528.087854] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3528.087856] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3528.087865] Call Trace: [ 3528.087871] [ 3528.087876] ? show_regs.cold.9+0x22/0x2f [ 3528.087886] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3528.087894] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3528.087898] ? watchdog+0x40/0x40 [ 3528.087902] ? hrtimer_interrupt+0x128/0x2c0 [ 3528.087905] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3528.087913] ? apic_timer_interrupt+0xf/0x20 [ 3528.087917] [ 3528.087919] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3528.087924] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3528.087928] do_raw_spin_lock+0x177/0x190 [ 3528.087932] _raw_spin_lock+0x21/0x30 [ 3528.087936] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3528.088159] process_one_work+0x2c8/0x700 [ 3528.088165] worker_thread+0x296/0x6e0 [ 3528.088169] ? rescuer_thread+0x570/0x570 [ 3528.088172] kthread+0x1d1/0x200 [ 3528.088177] ? set_kthread_struct+0x70/0x70 [ 3528.088181] ret_from_fork+0x1f/0x30 [ 3528.338392] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3528.345002] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3528.351317] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3528.365544] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3528.369553] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3528.383076] RDX: 0000000000003d56 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3528.385378] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3528.399962] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3528.408546] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3528.417506] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3528.426013] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3528.437328] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3528.449498] Call Trace: [ 3528.453172] [ 3528.456346] ? show_regs.cold.9+0x22/0x2f [ 3528.458374] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3528.463061] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3528.469293] ? watchdog+0x40/0x40 [ 3528.470572] ? hrtimer_interrupt+0x128/0x2c0 [ 3528.473961] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3528.476700] ? apic_timer_interrupt+0xf/0x20 [ 3528.479343] [ 3528.480763] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3528.483807] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3528.486970] do_raw_spin_lock+0x177/0x190 [ 3528.489833] _raw_spin_lock+0x21/0x30 [ 3528.491800] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3528.495363] obd_mod_free+0x46/0x140 [ptlrpc] [ 3528.500564] mdc_commit_open+0x91/0xb0 [mdc] [ 3528.503642] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3528.514144] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3528.520313] after_reply+0xe2c/0x1780 [ptlrpc] [ 3528.523242] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3528.535392] ? __next_timer_interrupt+0x160/0x160 [ 3528.539043] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3528.542395] ? do_wait_intr+0xf0/0xf0 [ 3528.544783] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3528.548321] kthread+0x1d1/0x200 [ 3528.550294] ? set_kthread_struct+0x70/0x70 [ 3528.552066] ret_from_fork+0x1f/0x30 [ 3552.059557] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3552.065570] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3552.107626] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3552.111763] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3552.124884] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3552.126650] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3552.139343] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3552.142158] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3552.145409] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3552.150741] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3552.153341] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3552.157483] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3552.160703] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3552.167770] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3552.170775] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3552.173579] Call Trace: [ 3552.174610] [ 3552.175463] ? show_regs.cold.9+0x22/0x2f [ 3552.178637] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3552.180406] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3552.185547] ? watchdog+0x40/0x40 [ 3552.189383] ? hrtimer_interrupt+0x128/0x2c0 [ 3552.191397] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3552.192908] ? apic_timer_interrupt+0xf/0x20 [ 3552.196085] [ 3552.197549] ? native_safe_halt+0x2e/0x50 [ 3552.199877] kvm_wait+0xcd/0xe0 [ 3552.201010] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3552.203264] do_raw_spin_lock+0x177/0x190 [ 3552.205666] _raw_spin_lock+0x21/0x30 [ 3552.206862] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3552.210164] ? do_raw_spin_unlock+0x75/0x190 [ 3552.211520] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3552.214824] ? do_wait_intr+0xf0/0xf0 [ 3552.216358] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3552.219485] mdc_reint+0x2d/0x110 [mdc] [ 3552.220853] mdc_unlink+0x658/0x870 [mdc] [ 3552.222464] lmv_unlink+0x34d/0xbd0 [lmv] [ 3552.224181] ll_unlink+0x24d/0x7f0 [lustre] [ 3552.225632] vfs_unlink+0x202/0x330 [ 3552.227273] do_unlinkat+0x2fa/0x460 [ 3552.228423] __x64_sys_unlink+0x37/0x50 [ 3552.229561] do_syscall_64+0xc1/0x440 [ 3552.231657] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3552.237814] RIP: 0033:0x7f33957fc10b [ 3552.240362] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3552.247541] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3552.250531] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3552.253735] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3552.257470] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3552.262433] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3552.267711] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3556.056087] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3556.058065] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3556.063696] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3556.063780] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3556.063787] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3556.063791] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3556.067308] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3556.103493] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3556.103509] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3556.103515] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3556.103517] RDX: 0000000000000bb4 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3556.103520] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3556.103522] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3556.103523] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3556.103526] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3556.103529] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3556.103531] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3556.103550] Call Trace: [ 3556.103556] [ 3556.103561] ? show_regs.cold.9+0x22/0x2f [ 3556.103574] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3556.103580] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3556.103584] ? watchdog+0x40/0x40 [ 3556.103588] ? hrtimer_interrupt+0x128/0x2c0 [ 3556.103593] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3556.103602] ? apic_timer_interrupt+0xf/0x20 [ 3556.103607] [ 3556.103610] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3556.103617] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3556.103621] do_raw_spin_lock+0x177/0x190 [ 3556.103626] _raw_spin_lock+0x21/0x30 [ 3556.103632] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3556.103923] obd_mod_free+0x46/0x140 [ptlrpc] [ 3556.104576] mdc_commit_open+0x91/0xb0 [mdc] [ 3556.104616] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3556.104814] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3556.105240] after_reply+0xe2c/0x1780 [ptlrpc] [ 3556.105664] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3556.105904] ? __next_timer_interrupt+0x160/0x160 [ 3556.105916] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3556.106579] ? do_wait_intr+0xf0/0xf0 [ 3556.106589] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3556.106840] kthread+0x1d1/0x200 [ 3556.106846] ? set_kthread_struct+0x70/0x70 [ 3556.106850] ret_from_fork+0x1f/0x30 [ 3556.326474] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3556.331518] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3556.336591] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3556.340232] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3556.343749] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3556.358730] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3556.368752] RAX: 0000000000003987 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3556.376279] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3556.387282] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3556.394922] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3556.409502] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3556.415544] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3556.422738] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3556.430707] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3556.441622] Call Trace: [ 3556.444764] [ 3556.447494] ? show_regs.cold.9+0x22/0x2f [ 3556.459073] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3556.466470] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3556.472013] ? watchdog+0x40/0x40 [ 3556.476007] ? hrtimer_interrupt+0x128/0x2c0 [ 3556.480820] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3556.484247] ? apic_timer_interrupt+0xf/0x20 [ 3556.487071] [ 3556.489038] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3556.498059] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3556.506053] do_raw_spin_lock+0x177/0x190 [ 3556.509042] _raw_spin_lock+0x21/0x30 [ 3556.513708] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3556.523164] process_one_work+0x2c8/0x700 [ 3556.524447] worker_thread+0x296/0x6e0 [ 3556.525327] ? rescuer_thread+0x570/0x570 [ 3556.534428] kthread+0x1d1/0x200 [ 3556.535603] ? set_kthread_struct+0x70/0x70 [ 3556.538960] ret_from_fork+0x1f/0x30 [ 3580.059120] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3580.066456] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3580.107559] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3580.115216] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3580.120266] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3580.122839] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3580.132528] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3580.136517] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3580.139998] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3580.143198] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3580.146198] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3580.149589] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3580.152858] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3580.156006] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3580.158226] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3580.161438] Call Trace: [ 3580.162355] [ 3580.163198] ? show_regs.cold.9+0x22/0x2f [ 3580.164841] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3580.166828] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3580.168497] ? watchdog+0x40/0x40 [ 3580.170068] ? hrtimer_interrupt+0x128/0x2c0 [ 3580.172047] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3580.173960] ? apic_timer_interrupt+0xf/0x20 [ 3580.175901] [ 3580.176788] ? native_safe_halt+0x2e/0x50 [ 3580.178433] kvm_wait+0xcd/0xe0 [ 3580.179885] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3580.181998] do_raw_spin_lock+0x177/0x190 [ 3580.183827] _raw_spin_lock+0x21/0x30 [ 3580.185457] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3580.188150] ? do_raw_spin_unlock+0x75/0x190 [ 3580.189890] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3580.192197] ? do_wait_intr+0xf0/0xf0 [ 3580.194431] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3580.196257] mdc_reint+0x2d/0x110 [mdc] [ 3580.198201] mdc_unlink+0x658/0x870 [mdc] [ 3580.199547] lmv_unlink+0x34d/0xbd0 [lmv] [ 3580.201617] ll_unlink+0x24d/0x7f0 [lustre] [ 3580.203772] vfs_unlink+0x202/0x330 [ 3580.205226] do_unlinkat+0x2fa/0x460 [ 3580.207520] __x64_sys_unlink+0x37/0x50 [ 3580.208980] do_syscall_64+0xc1/0x440 [ 3580.210564] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3580.212960] RIP: 0033:0x7f33957fc10b [ 3580.216613] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3580.233978] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3580.237194] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3580.239826] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3580.242929] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3580.245983] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3580.251527] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3584.055258] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3584.058947] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3584.071810] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3584.071886] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3584.071893] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3584.071896] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3584.072465] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3584.072469] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3584.072474] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3584.072477] RDX: 00000000000031b6 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3584.072479] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3584.072481] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3584.072483] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3584.072486] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3584.072489] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3584.072491] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3584.072499] Call Trace: [ 3584.072504] [ 3584.072510] ? show_regs.cold.9+0x22/0x2f [ 3584.072519] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3584.072526] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3584.072530] ? watchdog+0x40/0x40 [ 3584.072535] ? hrtimer_interrupt+0x128/0x2c0 [ 3584.072539] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3584.072548] ? apic_timer_interrupt+0xf/0x20 [ 3584.072564] [ 3584.072567] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3584.072571] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3584.072576] do_raw_spin_lock+0x177/0x190 [ 3584.072580] _raw_spin_lock+0x21/0x30 [ 3584.072585] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3584.074144] obd_mod_free+0x46/0x140 [ptlrpc] [ 3584.074364] mdc_commit_open+0x91/0xb0 [mdc] [ 3584.074403] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3584.076203] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3584.076434] after_reply+0xe2c/0x1780 [ptlrpc] [ 3584.077746] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3584.079628] ? __next_timer_interrupt+0x160/0x160 [ 3584.079641] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3584.079887] ? do_wait_intr+0xf0/0xf0 [ 3584.079896] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3584.088692] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3584.088773] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3584.088779] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3584.088786] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3584.089068] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3584.089078] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3584.089082] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3584.089087] RAX: 0000000000007e9f RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3584.089090] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3584.089092] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3584.089095] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3584.089098] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3584.089101] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3584.089103] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3584.089106] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3584.089115] Call Trace: [ 3584.089120] [ 3584.089125] ? show_regs.cold.9+0x22/0x2f [ 3584.089136] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3584.089142] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3584.089146] ? watchdog+0x40/0x40 [ 3584.089150] ? hrtimer_interrupt+0x128/0x2c0 [ 3584.089155] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3584.089164] ? apic_timer_interrupt+0xf/0x20 [ 3584.089169] [ 3584.089172] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3584.089177] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3584.089183] do_raw_spin_lock+0x177/0x190 [ 3584.089188] _raw_spin_lock+0x21/0x30 [ 3584.089193] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3584.089396] process_one_work+0x2c8/0x700 [ 3584.089404] worker_thread+0x296/0x6e0 [ 3584.089409] ? rescuer_thread+0x570/0x570 [ 3584.089413] kthread+0x1d1/0x200 [ 3584.089417] ? set_kthread_struct+0x70/0x70 [ 3584.089420] ret_from_fork+0x1f/0x30 [ 3584.576840] kthread+0x1d1/0x200 [ 3584.580450] ? set_kthread_struct+0x70/0x70 [ 3584.583401] ret_from_fork+0x1f/0x30 [ 3598.058302] rcu: INFO: rcu_sched self-detected stall on CPU [ 3598.062221] rcu: 0-....: (930947 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=281732 [ 3598.069124] (t=1500038 jiffies g=297885 q=8693) [ 3598.071501] NMI backtrace for cpu 0 [ 3598.073455] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3598.078781] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3598.082775] Call Trace: [ 3598.083906] [ 3598.085109] ? dump_stack+0xbb/0x10e [ 3598.086285] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3598.092344] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3598.095386] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3598.101144] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3598.106149] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3598.110111] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3598.115939] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3598.117906] ? update_process_times+0x6d/0xc0 [ 3598.121481] ? tick_sched_handle.isra.18+0x27/0x90 [ 3598.126471] ? tick_sched_timer+0x53/0xb0 [ 3598.128957] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3598.130908] ? tick_sched_handle.isra.18+0x90/0x90 [ 3598.132937] ? hrtimer_interrupt+0x128/0x2c0 [ 3598.135141] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3598.136876] ? apic_timer_interrupt+0xf/0x20 [ 3598.138758] [ 3598.139627] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3598.141855] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3598.143992] ? do_raw_spin_lock+0x177/0x190 [ 3598.149032] ? _raw_spin_lock+0x21/0x30 [ 3598.150720] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3598.157082] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3598.162599] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3598.164179] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3598.169261] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3598.171990] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3598.174149] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3598.176575] ? __next_timer_interrupt+0x160/0x160 [ 3598.178793] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3598.181898] ? do_wait_intr+0xf0/0xf0 [ 3598.183378] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3598.186234] ? kthread+0x1d1/0x200 [ 3598.187781] ? set_kthread_struct+0x70/0x70 [ 3598.189759] ? ret_from_fork+0x1f/0x30 [ 3608.059050] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3608.063945] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3608.102844] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3608.114030] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3608.121589] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30a/0x6d0 [ 3608.129221] Code: 03 00 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 90 83 e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 [ 3608.155835] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 3608.161857] RAX: 0000000000002f80 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3608.167883] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3608.171929] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3608.176464] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3608.179683] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3608.182904] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3608.186685] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3608.189412] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3608.192631] Call Trace: [ 3608.193847] [ 3608.194831] ? show_regs.cold.9+0x22/0x2f [ 3608.196724] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3608.198845] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3608.200874] ? watchdog+0x40/0x40 [ 3608.202468] ? hrtimer_interrupt+0x128/0x2c0 [ 3608.204274] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3608.206661] ? apic_timer_interrupt+0xf/0x20 [ 3608.208616] [ 3608.209840] ? __pv_queued_spin_lock_slowpath+0x30a/0x6d0 [ 3608.212442] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3608.216714] do_raw_spin_lock+0x177/0x190 [ 3608.221010] _raw_spin_lock+0x21/0x30 [ 3608.223700] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3608.227756] ? do_raw_spin_unlock+0x75/0x190 [ 3608.230987] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3608.233621] ? do_wait_intr+0xf0/0xf0 [ 3608.235626] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3608.238495] mdc_reint+0x2d/0x110 [mdc] [ 3608.242045] mdc_unlink+0x658/0x870 [mdc] [ 3608.243675] lmv_unlink+0x34d/0xbd0 [lmv] [ 3608.246574] ll_unlink+0x24d/0x7f0 [lustre] [ 3608.248795] vfs_unlink+0x202/0x330 [ 3608.251050] do_unlinkat+0x2fa/0x460 [ 3608.253736] __x64_sys_unlink+0x37/0x50 [ 3608.255927] do_syscall_64+0xc1/0x440 [ 3608.258522] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3608.263699] RIP: 0033:0x7f33957fc10b [ 3608.265966] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3608.286906] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3608.294423] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3608.299611] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3608.304415] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3608.310495] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3608.314609] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3612.058017] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3612.063419] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3612.127183] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3612.146898] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3612.151505] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3612.160329] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3612.167461] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3612.200248] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3612.211498] RAX: 0000000000003e63 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3612.222517] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3612.228351] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3612.237127] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3612.241407] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3612.245600] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3612.249680] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3612.253671] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3612.257316] Call Trace: [ 3612.258958] [ 3612.260362] ? show_regs.cold.9+0x22/0x2f [ 3612.262264] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3612.264440] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3612.265899] ? watchdog+0x40/0x40 [ 3612.267132] ? hrtimer_interrupt+0x128/0x2c0 [ 3612.270269] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3612.272483] ? apic_timer_interrupt+0xf/0x20 [ 3612.274925] [ 3612.276121] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3612.278940] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3612.282016] do_raw_spin_lock+0x177/0x190 [ 3612.284479] _raw_spin_lock+0x21/0x30 [ 3612.286543] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3612.292412] process_one_work+0x2c8/0x700 [ 3612.296597] worker_thread+0x296/0x6e0 [ 3612.298319] ? rescuer_thread+0x570/0x570 [ 3612.301119] kthread+0x1d1/0x200 [ 3612.302336] ? set_kthread_struct+0x70/0x70 [ 3612.304833] ret_from_fork+0x1f/0x30 [ 3624.053041] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3624.056477] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3624.076616] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3624.081879] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3624.084715] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3624.086949] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3624.095741] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3624.099173] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3624.102687] RDX: 000000000000381d RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3624.105876] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3624.109020] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3624.111548] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3624.114661] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3624.118524] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3624.121036] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3624.124003] Call Trace: [ 3624.124906] [ 3624.126025] ? show_regs.cold.9+0x22/0x2f [ 3624.127774] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3624.129668] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3624.131546] ? watchdog+0x40/0x40 [ 3624.133110] ? hrtimer_interrupt+0x128/0x2c0 [ 3624.134997] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3624.137047] ? apic_timer_interrupt+0xf/0x20 [ 3624.138937] [ 3624.139719] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3624.142002] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3624.144109] do_raw_spin_lock+0x177/0x190 [ 3624.145895] _raw_spin_lock+0x21/0x30 [ 3624.147516] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3624.150110] obd_mod_free+0x46/0x140 [ptlrpc] [ 3624.152367] mdc_commit_open+0x91/0xb0 [mdc] [ 3624.153844] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3624.156296] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3624.158902] after_reply+0xe2c/0x1780 [ptlrpc] [ 3624.160734] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3624.163297] ? __next_timer_interrupt+0x160/0x160 [ 3624.165362] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3624.167524] ? do_wait_intr+0xf0/0xf0 [ 3624.169142] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3624.171531] kthread+0x1d1/0x200 [ 3624.172987] ? set_kthread_struct+0x70/0x70 [ 3624.174926] ret_from_fork+0x1f/0x30 [ 3636.059028] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3636.061343] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3636.115919] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3636.123454] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3636.127853] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3636.131363] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3636.144457] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3636.152786] RAX: 0000000000000d98 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3636.158364] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3636.163308] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3636.166350] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3636.179388] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3636.184107] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3636.191367] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3636.201024] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3636.207387] Call Trace: [ 3636.215419] [ 3636.218220] ? show_regs.cold.9+0x22/0x2f [ 3636.227824] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3636.231207] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3636.243705] ? watchdog+0x40/0x40 [ 3636.250443] ? hrtimer_interrupt+0x128/0x2c0 [ 3636.254215] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3636.259215] ? apic_timer_interrupt+0xf/0x20 [ 3636.266165] [ 3636.269861] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3636.278596] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3636.285400] do_raw_spin_lock+0x177/0x190 [ 3636.292856] _raw_spin_lock+0x21/0x30 [ 3636.296176] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3636.304658] ? do_raw_spin_unlock+0x75/0x190 [ 3636.308405] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3636.315778] ? do_wait_intr+0xf0/0xf0 [ 3636.320899] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3636.327921] mdc_reint+0x2d/0x110 [mdc] [ 3636.334982] mdc_unlink+0x658/0x870 [mdc] [ 3636.343729] lmv_unlink+0x34d/0xbd0 [lmv] [ 3636.345237] ll_unlink+0x24d/0x7f0 [lustre] [ 3636.356314] vfs_unlink+0x202/0x330 [ 3636.365845] do_unlinkat+0x2fa/0x460 [ 3636.369127] __x64_sys_unlink+0x37/0x50 [ 3636.371961] do_syscall_64+0xc1/0x440 [ 3636.379454] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3636.384975] RIP: 0033:0x7f33957fc10b [ 3636.391150] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3636.428804] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3636.439634] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3636.449834] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3636.456543] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3636.465698] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3636.473045] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3640.059213] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 3640.066485] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3640.107171] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3640.113270] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3640.125375] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3640.129213] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3640.134918] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3640.144121] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3640.151086] RAX: 0000000000005728 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3640.155004] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3640.158696] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3640.166303] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3640.170790] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3640.174265] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3640.183642] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3640.189837] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3640.192999] Call Trace: [ 3640.194159] [ 3640.200038] ? show_regs.cold.9+0x22/0x2f [ 3640.204823] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3640.207551] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3640.210902] ? watchdog+0x40/0x40 [ 3640.213010] ? hrtimer_interrupt+0x128/0x2c0 [ 3640.214880] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3640.219442] ? apic_timer_interrupt+0xf/0x20 [ 3640.223365] [ 3640.229837] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3640.231857] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3640.237357] do_raw_spin_lock+0x177/0x190 [ 3640.238739] _raw_spin_lock+0x21/0x30 [ 3640.242338] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3640.249980] process_one_work+0x2c8/0x700 [ 3640.252830] worker_thread+0x296/0x6e0 [ 3640.263515] ? rescuer_thread+0x570/0x570 [ 3640.269423] kthread+0x1d1/0x200 [ 3640.271887] ? set_kthread_struct+0x70/0x70 [ 3640.274086] ret_from_fork+0x1f/0x30 [ 3652.053187] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3652.059875] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3652.091189] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3652.098356] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3652.103409] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3652.106267] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3652.117528] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3652.119963] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3652.124439] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3652.129194] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3652.132415] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3652.135309] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3652.140234] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3652.144154] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3652.146540] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3652.149839] Call Trace: [ 3652.150861] [ 3652.151708] ? show_regs.cold.9+0x22/0x2f [ 3652.153430] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3652.155716] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3652.157665] ? watchdog+0x40/0x40 [ 3652.159683] ? hrtimer_interrupt+0x128/0x2c0 [ 3652.161719] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3652.163910] ? apic_timer_interrupt+0xf/0x20 [ 3652.165612] [ 3652.166729] ? native_safe_halt+0x2e/0x50 [ 3652.169588] kvm_wait+0xcd/0xe0 [ 3652.171600] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3652.174081] do_raw_spin_lock+0x177/0x190 [ 3652.176263] _raw_spin_lock+0x21/0x30 [ 3652.178612] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3652.181777] obd_mod_free+0x46/0x140 [ptlrpc] [ 3652.184828] mdc_commit_open+0x91/0xb0 [mdc] [ 3652.187203] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3652.189453] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3652.191929] after_reply+0xe2c/0x1780 [ptlrpc] [ 3652.202666] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3652.207159] ? __next_timer_interrupt+0x160/0x160 [ 3652.211152] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3652.217072] ? do_wait_intr+0xf0/0xf0 [ 3652.221811] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3652.229712] kthread+0x1d1/0x200 [ 3652.233824] ? set_kthread_struct+0x70/0x70 [ 3652.235889] ret_from_fork+0x1f/0x30 [ 3664.059633] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3664.064359] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3664.095476] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3664.102886] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3664.106649] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3664.109193] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3664.115918] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3664.118276] RAX: 0000000000003b49 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3664.121466] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3664.124109] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3664.126166] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3664.128170] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3664.131169] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3664.134092] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3664.136519] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3664.142457] Call Trace: [ 3664.143512] [ 3664.144710] ? show_regs.cold.9+0x22/0x2f [ 3664.148173] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3664.149776] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3664.152257] ? watchdog+0x40/0x40 [ 3664.154215] ? hrtimer_interrupt+0x128/0x2c0 [ 3664.155808] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3664.159548] ? apic_timer_interrupt+0xf/0x20 [ 3664.160806] [ 3664.161489] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3664.167943] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3664.173954] do_raw_spin_lock+0x177/0x190 [ 3664.176077] _raw_spin_lock+0x21/0x30 [ 3664.178354] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3664.181861] ? do_raw_spin_unlock+0x75/0x190 [ 3664.184357] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3664.189804] ? do_wait_intr+0xf0/0xf0 [ 3664.191633] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3664.194385] mdc_reint+0x2d/0x110 [mdc] [ 3664.195839] mdc_unlink+0x658/0x870 [mdc] [ 3664.198719] lmv_unlink+0x34d/0xbd0 [lmv] [ 3664.200719] ll_unlink+0x24d/0x7f0 [lustre] [ 3664.203690] vfs_unlink+0x202/0x330 [ 3664.205564] do_unlinkat+0x2fa/0x460 [ 3664.207372] __x64_sys_unlink+0x37/0x50 [ 3664.208694] do_syscall_64+0xc1/0x440 [ 3664.210493] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3664.212548] RIP: 0033:0x7f33957fc10b [ 3664.214184] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3664.230903] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3664.234011] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3664.238496] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3664.241719] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3664.244805] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3664.253674] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3668.058011] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 3668.063635] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3668.100679] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3668.109972] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3668.114262] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3668.116963] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3668.119358] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3668.133493] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3668.140608] RAX: 0000000000007f17 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3668.148239] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3668.154166] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3668.157370] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3668.162961] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3668.169769] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3668.173246] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3668.175951] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3668.180407] Call Trace: [ 3668.183172] [ 3668.183954] ? show_regs.cold.9+0x22/0x2f [ 3668.191655] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3668.192846] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3668.197744] ? watchdog+0x40/0x40 [ 3668.199814] ? hrtimer_interrupt+0x128/0x2c0 [ 3668.203671] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3668.205291] ? apic_timer_interrupt+0xf/0x20 [ 3668.211636] [ 3668.214653] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3668.223115] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3668.228172] do_raw_spin_lock+0x177/0x190 [ 3668.233213] _raw_spin_lock+0x21/0x30 [ 3668.234485] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3668.243706] process_one_work+0x2c8/0x700 [ 3668.248712] worker_thread+0x296/0x6e0 [ 3668.252618] ? rescuer_thread+0x570/0x570 [ 3668.261082] kthread+0x1d1/0x200 [ 3668.262733] ? set_kthread_struct+0x70/0x70 [ 3668.270694] ret_from_fork+0x1f/0x30 [ 3680.053451] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3680.060811] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3680.114931] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3680.124960] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3680.135094] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3680.141745] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3680.158190] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3680.162923] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3680.168144] RDX: 0000000000007635 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3680.173770] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3680.180310] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3680.186377] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3680.191719] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3680.198039] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3680.204131] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3680.210840] Call Trace: [ 3680.212708] [ 3680.215480] ? show_regs.cold.9+0x22/0x2f [ 3680.219948] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3680.225112] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3680.228570] ? watchdog+0x40/0x40 [ 3680.233115] ? hrtimer_interrupt+0x128/0x2c0 [ 3680.238052] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3680.241732] ? apic_timer_interrupt+0xf/0x20 [ 3680.246068] [ 3680.247315] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3680.252978] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3680.255589] do_raw_spin_lock+0x177/0x190 [ 3680.260885] _raw_spin_lock+0x21/0x30 [ 3680.263149] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3680.265970] obd_mod_free+0x46/0x140 [ptlrpc] [ 3680.268150] mdc_commit_open+0x91/0xb0 [mdc] [ 3680.269873] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3680.272086] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3680.274483] after_reply+0xe2c/0x1780 [ptlrpc] [ 3680.277272] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3680.279903] ? __next_timer_interrupt+0x160/0x160 [ 3680.281736] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3680.283780] ? do_wait_intr+0xf0/0xf0 [ 3680.285607] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3680.287939] kthread+0x1d1/0x200 [ 3680.290609] ? set_kthread_struct+0x70/0x70 [ 3680.292200] ret_from_fork+0x1f/0x30 [ 3692.059278] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3692.064165] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3692.094324] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3692.100174] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3692.104877] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3692.107777] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3692.123595] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3692.126099] RAX: 0000000000005a37 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3692.131722] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3692.136240] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3692.143609] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3692.147784] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3692.151037] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3692.155130] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3692.158695] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3692.162076] Call Trace: [ 3692.165937] [ 3692.167002] ? show_regs.cold.9+0x22/0x2f [ 3692.170170] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3692.172930] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3692.175083] ? watchdog+0x40/0x40 [ 3692.176687] ? hrtimer_interrupt+0x128/0x2c0 [ 3692.179562] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3692.181969] ? apic_timer_interrupt+0xf/0x20 [ 3692.183757] [ 3692.184604] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3692.186610] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3692.189386] do_raw_spin_lock+0x177/0x190 [ 3692.190705] _raw_spin_lock+0x21/0x30 [ 3692.194904] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3692.199346] ? do_raw_spin_unlock+0x75/0x190 [ 3692.201627] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3692.206368] ? do_wait_intr+0xf0/0xf0 [ 3692.208187] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3692.211693] mdc_reint+0x2d/0x110 [mdc] [ 3692.212893] mdc_unlink+0x658/0x870 [mdc] [ 3692.217046] lmv_unlink+0x34d/0xbd0 [lmv] [ 3692.218925] ll_unlink+0x24d/0x7f0 [lustre] [ 3692.222932] vfs_unlink+0x202/0x330 [ 3692.224703] do_unlinkat+0x2fa/0x460 [ 3692.226616] __x64_sys_unlink+0x37/0x50 [ 3692.228053] do_syscall_64+0xc1/0x440 [ 3692.230059] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3692.233385] RIP: 0033:0x7f33957fc10b [ 3692.235654] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3692.246733] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3692.250592] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3692.258576] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3692.265991] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3692.272382] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3692.276776] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3696.059935] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 3696.075664] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3696.110638] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3696.128632] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3696.143738] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3696.156360] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3696.165788] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3696.193138] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3696.205955] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3696.218971] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3696.226819] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3696.234603] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3696.239454] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3696.256682] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3696.268640] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3696.272523] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3696.281624] Call Trace: [ 3696.283226] [ 3696.285168] ? show_regs.cold.9+0x22/0x2f [ 3696.289404] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3696.291164] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3696.300158] ? watchdog+0x40/0x40 [ 3696.304851] ? hrtimer_interrupt+0x128/0x2c0 [ 3696.307087] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3696.315413] ? apic_timer_interrupt+0xf/0x20 [ 3696.320250] [ 3696.321391] ? native_safe_halt+0x2e/0x50 [ 3696.325163] kvm_wait+0xcd/0xe0 [ 3696.327469] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3696.332616] do_raw_spin_lock+0x177/0x190 [ 3696.333800] _raw_spin_lock+0x21/0x30 [ 3696.335180] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3696.338177] process_one_work+0x2c8/0x700 [ 3696.340273] worker_thread+0x296/0x6e0 [ 3696.342301] ? rescuer_thread+0x570/0x570 [ 3696.354055] kthread+0x1d1/0x200 [ 3696.355364] ? set_kthread_struct+0x70/0x70 [ 3696.357291] ret_from_fork+0x1f/0x30 [ 3708.053761] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3708.064992] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3708.106409] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3708.118607] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3708.131887] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3708.137298] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3708.161056] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3708.170098] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3708.175563] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3708.180452] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3708.183979] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3708.188315] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3708.191464] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3708.195010] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3708.197506] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3708.201207] Call Trace: [ 3708.203334] [ 3708.205330] ? show_regs.cold.9+0x22/0x2f [ 3708.208172] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3708.215593] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3708.219309] ? watchdog+0x40/0x40 [ 3708.221619] ? hrtimer_interrupt+0x128/0x2c0 [ 3708.226467] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3708.230585] ? apic_timer_interrupt+0xf/0x20 [ 3708.236895] [ 3708.237997] ? native_safe_halt+0x2e/0x50 [ 3708.242178] kvm_wait+0xcd/0xe0 [ 3708.247051] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3708.249765] do_raw_spin_lock+0x177/0x190 [ 3708.254671] _raw_spin_lock+0x21/0x30 [ 3708.256648] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3708.264499] obd_mod_free+0x46/0x140 [ptlrpc] [ 3708.268608] mdc_commit_open+0x91/0xb0 [mdc] [ 3708.272786] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3708.279548] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3708.286500] after_reply+0xe2c/0x1780 [ptlrpc] [ 3708.290418] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3708.294698] ? __next_timer_interrupt+0x160/0x160 [ 3708.298585] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3708.301568] ? do_wait_intr+0xf0/0xf0 [ 3708.303412] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3708.310628] kthread+0x1d1/0x200 [ 3708.312355] ? set_kthread_struct+0x70/0x70 [ 3708.317380] ret_from_fork+0x1f/0x30 [ 3720.060521] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3720.068472] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3720.129485] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3720.146652] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3720.158927] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3720.163960] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3720.193490] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3720.204772] RAX: 0000000000002489 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3720.210415] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3720.216151] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3720.226903] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3720.234702] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3720.247134] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3720.256648] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3720.260583] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3720.265205] Call Trace: [ 3720.270087] [ 3720.274018] ? show_regs.cold.9+0x22/0x2f [ 3720.277053] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3720.281533] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3720.284950] ? watchdog+0x40/0x40 [ 3720.288240] ? hrtimer_interrupt+0x128/0x2c0 [ 3720.296772] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3720.303183] ? apic_timer_interrupt+0xf/0x20 [ 3720.308140] [ 3720.309446] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3720.313163] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3720.315978] do_raw_spin_lock+0x177/0x190 [ 3720.320580] _raw_spin_lock+0x21/0x30 [ 3720.323075] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3720.325743] ? do_raw_spin_unlock+0x75/0x190 [ 3720.328177] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3720.336528] ? do_wait_intr+0xf0/0xf0 [ 3720.339070] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3720.349163] mdc_reint+0x2d/0x110 [mdc] [ 3720.353268] mdc_unlink+0x658/0x870 [mdc] [ 3720.358469] lmv_unlink+0x34d/0xbd0 [lmv] [ 3720.361749] ll_unlink+0x24d/0x7f0 [lustre] [ 3720.366646] vfs_unlink+0x202/0x330 [ 3720.371038] do_unlinkat+0x2fa/0x460 [ 3720.375821] __x64_sys_unlink+0x37/0x50 [ 3720.379513] do_syscall_64+0xc1/0x440 [ 3720.384775] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3720.389788] RIP: 0033:0x7f33957fc10b [ 3720.392269] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3720.421719] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3720.431530] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3720.437701] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3720.440726] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3720.447509] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3720.452167] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3724.061900] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 3724.069119] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3724.121477] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3724.137307] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3724.145054] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3724.157844] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3724.160978] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3724.184143] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3724.189547] RAX: 000000000000788d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3724.193943] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3724.196825] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3724.202451] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3724.205623] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3724.209647] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3724.215520] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3724.217313] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3724.221132] Call Trace: [ 3724.222162] [ 3724.223162] ? show_regs.cold.9+0x22/0x2f [ 3724.224703] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3724.228183] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3724.230245] ? watchdog+0x40/0x40 [ 3724.231764] ? hrtimer_interrupt+0x128/0x2c0 [ 3724.233458] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3724.235167] ? apic_timer_interrupt+0xf/0x20 [ 3724.236740] [ 3724.237719] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3724.240285] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3724.243256] do_raw_spin_lock+0x177/0x190 [ 3724.245036] _raw_spin_lock+0x21/0x30 [ 3724.247625] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3724.251741] process_one_work+0x2c8/0x700 [ 3724.254152] worker_thread+0x296/0x6e0 [ 3724.255879] ? rescuer_thread+0x570/0x570 [ 3724.258289] kthread+0x1d1/0x200 [ 3724.260664] ? set_kthread_struct+0x70/0x70 [ 3724.262869] ret_from_fork+0x1f/0x30 [ 3736.053794] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3736.057446] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3736.082104] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3736.087041] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3736.091208] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3736.094010] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3736.102698] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3736.105700] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3736.109474] RDX: 0000000000007ebe RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3736.114009] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3736.117331] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3736.120814] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3736.124458] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3736.135232] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3736.137117] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3736.140028] Call Trace: [ 3736.142308] [ 3736.144077] ? show_regs.cold.9+0x22/0x2f [ 3736.146588] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3736.150813] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3736.153080] ? watchdog+0x40/0x40 [ 3736.154147] ? hrtimer_interrupt+0x128/0x2c0 [ 3736.156124] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3736.158360] ? apic_timer_interrupt+0xf/0x20 [ 3736.160313] [ 3736.161131] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3736.163720] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3736.167938] do_raw_spin_lock+0x177/0x190 [ 3736.170659] _raw_spin_lock+0x21/0x30 [ 3736.175098] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3736.179906] obd_mod_free+0x46/0x140 [ptlrpc] [ 3736.182691] mdc_commit_open+0x91/0xb0 [mdc] [ 3736.185100] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3736.187912] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3736.191907] after_reply+0xe2c/0x1780 [ptlrpc] [ 3736.194790] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3736.203741] ? __next_timer_interrupt+0x160/0x160 [ 3736.207812] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3736.211877] ? do_wait_intr+0xf0/0xf0 [ 3736.214892] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3736.219802] kthread+0x1d1/0x200 [ 3736.220840] ? set_kthread_struct+0x70/0x70 [ 3736.223894] ret_from_fork+0x1f/0x30 [ 3748.059058] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3748.064686] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3748.112803] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3748.125056] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3748.136158] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3748.138475] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3748.152547] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3748.157913] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3748.163019] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3748.169405] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3748.185541] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3748.192578] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3748.198520] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3748.207986] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3748.215325] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3748.232681] Call Trace: [ 3748.238640] [ 3748.246697] ? show_regs.cold.9+0x22/0x2f [ 3748.248123] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3748.257779] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3748.260127] ? watchdog+0x40/0x40 [ 3748.261483] ? hrtimer_interrupt+0x128/0x2c0 [ 3748.265215] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3748.271808] ? apic_timer_interrupt+0xf/0x20 [ 3748.274100] [ 3748.275131] ? native_safe_halt+0x2e/0x50 [ 3748.276990] kvm_wait+0xcd/0xe0 [ 3748.278750] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3748.281385] do_raw_spin_lock+0x177/0x190 [ 3748.286818] _raw_spin_lock+0x21/0x30 [ 3748.288385] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3748.292600] ? do_raw_spin_unlock+0x75/0x190 [ 3748.294226] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3748.296297] ? do_wait_intr+0xf0/0xf0 [ 3748.298629] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3748.301887] mdc_reint+0x2d/0x110 [mdc] [ 3748.303872] mdc_unlink+0x658/0x870 [mdc] [ 3748.306371] lmv_unlink+0x34d/0xbd0 [lmv] [ 3748.308081] ll_unlink+0x24d/0x7f0 [lustre] [ 3748.310002] vfs_unlink+0x202/0x330 [ 3748.312683] do_unlinkat+0x2fa/0x460 [ 3748.317802] __x64_sys_unlink+0x37/0x50 [ 3748.319252] do_syscall_64+0xc1/0x440 [ 3748.321894] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3748.323753] RIP: 0033:0x7f33957fc10b [ 3748.325635] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3748.334940] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3748.338805] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3748.342663] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3748.350183] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3748.353525] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3748.360102] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3752.058024] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3752.064024] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3752.090985] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3752.103985] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3752.119180] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3752.127577] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3752.137233] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3752.155256] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3752.164450] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3752.172204] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3752.184417] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3752.193808] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3752.198389] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3752.210640] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3752.216668] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3752.225556] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3752.229721] Call Trace: [ 3752.231464] [ 3752.232867] ? show_regs.cold.9+0x22/0x2f [ 3752.234928] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3752.237369] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3752.241658] ? watchdog+0x40/0x40 [ 3752.246961] ? hrtimer_interrupt+0x128/0x2c0 [ 3752.251115] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3752.257883] ? apic_timer_interrupt+0xf/0x20 [ 3752.267547] [ 3752.268715] ? native_safe_halt+0x2e/0x50 [ 3752.270741] kvm_wait+0xcd/0xe0 [ 3752.275872] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3752.278834] do_raw_spin_lock+0x177/0x190 [ 3752.283756] _raw_spin_lock+0x21/0x30 [ 3752.288746] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3752.297296] process_one_work+0x2c8/0x700 [ 3752.300873] worker_thread+0x296/0x6e0 [ 3752.304331] ? rescuer_thread+0x570/0x570 [ 3752.306772] kthread+0x1d1/0x200 [ 3752.311612] ? set_kthread_struct+0x70/0x70 [ 3752.313823] ret_from_fork+0x1f/0x30 [ 3764.054763] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3764.061600] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3764.126596] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3764.148174] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3764.160470] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3764.169587] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3764.196004] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3764.208596] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3764.223638] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3764.232696] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3764.240550] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3764.250666] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3764.255727] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3764.261182] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3764.268513] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3764.275924] Call Trace: [ 3764.277562] [ 3764.281378] ? show_regs.cold.9+0x22/0x2f [ 3764.285115] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3764.288907] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3764.296335] ? watchdog+0x40/0x40 [ 3764.300326] ? hrtimer_interrupt+0x128/0x2c0 [ 3764.304054] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3764.308966] ? apic_timer_interrupt+0xf/0x20 [ 3764.311722] [ 3764.322548] ? native_safe_halt+0x2e/0x50 [ 3764.324272] kvm_wait+0xcd/0xe0 [ 3764.330971] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3764.333914] do_raw_spin_lock+0x177/0x190 [ 3764.336748] _raw_spin_lock+0x21/0x30 [ 3764.341607] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3764.349333] obd_mod_free+0x46/0x140 [ptlrpc] [ 3764.351813] mdc_commit_open+0x91/0xb0 [mdc] [ 3764.359122] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3764.365720] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3764.369674] after_reply+0xe2c/0x1780 [ptlrpc] [ 3764.374355] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3764.381931] ? __next_timer_interrupt+0x160/0x160 [ 3764.384839] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3764.390518] ? do_wait_intr+0xf0/0xf0 [ 3764.394769] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3764.403990] kthread+0x1d1/0x200 [ 3764.407827] ? set_kthread_struct+0x70/0x70 [ 3764.412307] ret_from_fork+0x1f/0x30 [ 3776.059149] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3776.061460] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3776.090667] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3776.104782] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3776.110573] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3776.114640] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3776.123062] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3776.130922] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3776.133242] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3776.137234] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3776.140485] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3776.146960] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3776.150298] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3776.154611] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3776.160713] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3776.168354] Call Trace: [ 3776.169595] [ 3776.171350] ? show_regs.cold.9+0x22/0x2f [ 3776.174330] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3776.178483] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3776.189578] ? watchdog+0x40/0x40 [ 3776.191081] ? hrtimer_interrupt+0x128/0x2c0 [ 3776.192401] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3776.198393] ? apic_timer_interrupt+0xf/0x20 [ 3776.199810] [ 3776.200886] ? native_safe_halt+0x2e/0x50 [ 3776.203560] kvm_wait+0xcd/0xe0 [ 3776.206699] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3776.211684] do_raw_spin_lock+0x177/0x190 [ 3776.214452] _raw_spin_lock+0x21/0x30 [ 3776.216898] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3776.226250] ? do_raw_spin_unlock+0x75/0x190 [ 3776.228992] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3776.230844] ? do_wait_intr+0xf0/0xf0 [ 3776.237516] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3776.242527] mdc_reint+0x2d/0x110 [mdc] [ 3776.244623] mdc_unlink+0x658/0x870 [mdc] [ 3776.246262] lmv_unlink+0x34d/0xbd0 [lmv] [ 3776.248640] ll_unlink+0x24d/0x7f0 [lustre] [ 3776.254353] vfs_unlink+0x202/0x330 [ 3776.257236] do_unlinkat+0x2fa/0x460 [ 3776.260184] __x64_sys_unlink+0x37/0x50 [ 3776.264089] do_syscall_64+0xc1/0x440 [ 3776.267586] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3776.269258] RIP: 0033:0x7f33957fc10b [ 3776.273330] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3776.289227] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3776.296195] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3776.303119] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3776.312076] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3776.318964] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3776.326675] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3778.061536] rcu: INFO: rcu_sched self-detected stall on CPU [ 3778.067727] rcu: 0-....: (1044874 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=316018 [ 3778.079255] (t=1680041 jiffies g=297885 q=9341) [ 3778.081627] NMI backtrace for cpu 0 [ 3778.083426] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3778.088834] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3778.099027] Call Trace: [ 3778.105512] [ 3778.107011] ? dump_stack+0xbb/0x10e [ 3778.109160] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3778.111909] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3778.114012] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3778.116191] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3778.119283] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3778.124253] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3778.129508] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3778.132478] ? update_process_times+0x6d/0xc0 [ 3778.135153] ? tick_sched_handle.isra.18+0x27/0x90 [ 3778.138170] ? tick_sched_timer+0x53/0xb0 [ 3778.140322] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3778.144314] ? tick_sched_handle.isra.18+0x90/0x90 [ 3778.148978] ? hrtimer_interrupt+0x128/0x2c0 [ 3778.151361] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3778.157121] ? apic_timer_interrupt+0xf/0x20 [ 3778.159463] [ 3778.160849] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3778.164619] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3778.170448] ? do_raw_spin_lock+0x177/0x190 [ 3778.173825] ? _raw_spin_lock+0x21/0x30 [ 3778.176488] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3778.182096] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3778.187433] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3778.194934] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3778.201706] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3778.213446] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3778.217379] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3778.220256] ? __next_timer_interrupt+0x160/0x160 [ 3778.222064] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3778.224919] ? do_wait_intr+0xf0/0xf0 [ 3778.228812] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3778.230865] ? kthread+0x1d1/0x200 [ 3778.233380] ? set_kthread_struct+0x70/0x70 [ 3778.235609] ? ret_from_fork+0x1f/0x30 [ 3780.061598] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3780.070359] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3780.118111] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3780.122514] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3780.126435] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3780.128871] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3780.132170] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3780.143106] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 3780.146651] RAX: 0000000000005e1a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3780.149990] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3780.153259] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3780.156327] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3780.159193] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3780.162456] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3780.166765] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3780.168583] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3780.171582] Call Trace: [ 3780.173520] [ 3780.174838] ? show_regs.cold.9+0x22/0x2f [ 3780.177198] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3780.180331] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3780.182338] ? watchdog+0x40/0x40 [ 3780.183614] ? hrtimer_interrupt+0x128/0x2c0 [ 3780.185409] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3780.187012] ? apic_timer_interrupt+0xf/0x20 [ 3780.188593] [ 3780.189385] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3780.191766] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3780.194007] do_raw_spin_lock+0x177/0x190 [ 3780.195502] _raw_spin_lock+0x21/0x30 [ 3780.198231] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3780.203594] process_one_work+0x2c8/0x700 [ 3780.208147] worker_thread+0x296/0x6e0 [ 3780.210626] ? rescuer_thread+0x570/0x570 [ 3780.212126] kthread+0x1d1/0x200 [ 3780.215406] ? set_kthread_struct+0x70/0x70 [ 3780.218459] ret_from_fork+0x1f/0x30 [ 3804.053342] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3804.059201] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3804.065966] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) [ 3804.072883] Modules linked in: lustre(O) [ 3804.085110] libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3804.085171] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3804.085263] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3804.085268] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3804.085288] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3804.085292] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3804.085299] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3804.085302] RDX: 00000000000004ea RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3804.085304] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3804.085306] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3804.085308] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3804.085311] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3804.085314] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3804.085316] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3804.085328] Call Trace: [ 3804.085333] [ 3804.085338] ? show_regs.cold.9+0x22/0x2f [ 3804.085413] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3804.085420] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3804.085427] ? watchdog+0x40/0x40 [ 3804.085432] ? hrtimer_interrupt+0x128/0x2c0 [ 3804.085436] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3804.085445] ? apic_timer_interrupt+0xf/0x20 [ 3804.085450] [ 3804.085453] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3804.085457] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3804.085461] do_raw_spin_lock+0x177/0x190 [ 3804.085465] _raw_spin_lock+0x21/0x30 [ 3804.085472] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3804.085819] obd_mod_free+0x46/0x140 [ptlrpc] [ 3804.086435] mdc_commit_open+0x91/0xb0 [mdc] [ 3804.086472] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3804.086722] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3804.086971] after_reply+0xe2c/0x1780 [ptlrpc] [ 3804.089086] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss [ 3804.104780] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3804.115873] nfsv4 [ 3804.119887] ? __next_timer_interrupt+0x160/0x160 [ 3804.119911] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3804.120370] ? do_wait_intr+0xf0/0xf0 [ 3804.120380] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3804.120879] kthread+0x1d1/0x200 [ 3804.120886] ? set_kthread_struct+0x70/0x70 [ 3804.120894] ret_from_fork+0x1f/0x30 [ 3804.311600] dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3804.324565] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3804.336161] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3804.340764] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3804.343391] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3804.354402] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3804.357701] RAX: 0000000000002573 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3804.361348] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3804.365229] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3804.368390] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3804.373732] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3804.377362] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3804.380907] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3804.383918] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3804.387521] Call Trace: [ 3804.392477] [ 3804.393386] ? show_regs.cold.9+0x22/0x2f [ 3804.395305] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3804.398030] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3804.403911] ? watchdog+0x40/0x40 [ 3804.405575] ? hrtimer_interrupt+0x128/0x2c0 [ 3804.414905] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3804.419242] ? apic_timer_interrupt+0xf/0x20 [ 3804.420475] [ 3804.423125] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3804.425165] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3804.432160] do_raw_spin_lock+0x177/0x190 [ 3804.434710] _raw_spin_lock+0x21/0x30 [ 3804.436745] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3804.439144] ? do_raw_spin_unlock+0x75/0x190 [ 3804.444677] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3804.447140] ? do_wait_intr+0xf0/0xf0 [ 3804.448585] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3804.454102] mdc_reint+0x2d/0x110 [mdc] [ 3804.457827] mdc_unlink+0x658/0x870 [mdc] [ 3804.461110] lmv_unlink+0x34d/0xbd0 [lmv] [ 3804.465318] ll_unlink+0x24d/0x7f0 [lustre] [ 3804.468921] vfs_unlink+0x202/0x330 [ 3804.470457] do_unlinkat+0x2fa/0x460 [ 3804.472072] __x64_sys_unlink+0x37/0x50 [ 3804.473708] do_syscall_64+0xc1/0x440 [ 3804.475475] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3804.479362] RIP: 0033:0x7f33957fc10b [ 3804.481191] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3804.496160] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3804.500946] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3804.505246] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3804.511436] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3804.514577] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3804.518160] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3808.058029] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3808.065323] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3808.107858] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3808.113965] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3808.122636] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3808.127217] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3808.128783] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3808.143940] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3808.146606] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3808.149394] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3808.154525] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3808.157969] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3808.162957] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3808.166521] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3808.172408] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3808.176805] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3808.180891] Call Trace: [ 3808.181695] [ 3808.182332] ? show_regs.cold.9+0x22/0x2f [ 3808.192125] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3808.193581] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3808.196494] ? watchdog+0x40/0x40 [ 3808.198722] ? hrtimer_interrupt+0x128/0x2c0 [ 3808.201894] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3808.212574] ? apic_timer_interrupt+0xf/0x20 [ 3808.214349] [ 3808.215043] ? native_safe_halt+0x2e/0x50 [ 3808.219368] kvm_wait+0xcd/0xe0 [ 3808.223672] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3808.226470] do_raw_spin_lock+0x177/0x190 [ 3808.228291] _raw_spin_lock+0x21/0x30 [ 3808.232614] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3808.238986] process_one_work+0x2c8/0x700 [ 3808.242882] worker_thread+0x296/0x6e0 [ 3808.247843] ? rescuer_thread+0x570/0x570 [ 3808.251466] kthread+0x1d1/0x200 [ 3808.253876] ? set_kthread_struct+0x70/0x70 [ 3808.256772] ret_from_fork+0x1f/0x30 [ 3832.053196] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3832.058378] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3832.059005] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3832.104070] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3832.104083] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3832.104087] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3832.104101] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3832.104106] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3832.104114] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3832.104116] RDX: 0000000000001e62 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3832.104119] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3832.104121] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3832.104123] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3832.104126] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3832.104129] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3832.104142] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3832.104150] Call Trace: [ 3832.104156] [ 3832.104161] ? show_regs.cold.9+0x22/0x2f [ 3832.104171] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3832.104177] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3832.104181] ? watchdog+0x40/0x40 [ 3832.104186] ? hrtimer_interrupt+0x128/0x2c0 [ 3832.104190] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3832.104199] ? apic_timer_interrupt+0xf/0x20 [ 3832.104204] [ 3832.104207] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3832.104212] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3832.104217] do_raw_spin_lock+0x177/0x190 [ 3832.104222] _raw_spin_lock+0x21/0x30 [ 3832.104228] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3832.112260] Modules linked in: [ 3832.120711] obd_mod_free+0x46/0x140 [ptlrpc] [ 3832.121022] mdc_commit_open+0x91/0xb0 [mdc] [ 3832.121069] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3832.121596] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3832.132629] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) [ 3832.137839] after_reply+0xe2c/0x1780 [ptlrpc] [ 3832.164906] lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3832.164977] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3832.164985] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3832.164988] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30a/0x6d0 [ 3832.165003] Code: 03 00 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 90 83 e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 [ 3832.165008] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3832.165012] RAX: 0000000000006890 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3832.165015] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3832.165017] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3832.165020] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3832.165022] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3832.165025] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3832.165029] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3832.165031] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3832.165040] Call Trace: [ 3832.165047] [ 3832.165052] ? show_regs.cold.9+0x22/0x2f [ 3832.165074] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3832.165082] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3832.165086] ? watchdog+0x40/0x40 [ 3832.165090] ? hrtimer_interrupt+0x128/0x2c0 [ 3832.165094] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3832.165103] ? apic_timer_interrupt+0xf/0x20 [ 3832.165108] [ 3832.165110] ? __pv_queued_spin_lock_slowpath+0x30a/0x6d0 [ 3832.165115] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3832.165120] do_raw_spin_lock+0x177/0x190 [ 3832.165125] _raw_spin_lock+0x21/0x30 [ 3832.165130] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3832.169160] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3832.176269] ? do_raw_spin_unlock+0x75/0x190 [ 3832.176290] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3832.181316] ? __next_timer_interrupt+0x160/0x160 [ 3832.181344] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3832.181854] ? do_wait_intr+0xf0/0xf0 [ 3832.181866] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3832.182112] kthread+0x1d1/0x200 [ 3832.182118] ? set_kthread_struct+0x70/0x70 [ 3832.182441] ret_from_fork+0x1f/0x30 [ 3832.472499] ? do_wait_intr+0xf0/0xf0 [ 3832.473840] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3832.476222] mdc_reint+0x2d/0x110 [mdc] [ 3832.478509] mdc_unlink+0x658/0x870 [mdc] [ 3832.484284] lmv_unlink+0x34d/0xbd0 [lmv] [ 3832.490098] ll_unlink+0x24d/0x7f0 [lustre] [ 3832.493606] vfs_unlink+0x202/0x330 [ 3832.494867] do_unlinkat+0x2fa/0x460 [ 3832.498585] __x64_sys_unlink+0x37/0x50 [ 3832.505439] do_syscall_64+0xc1/0x440 [ 3832.509609] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3832.511777] RIP: 0033:0x7f33957fc10b [ 3832.513788] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3832.531462] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3832.534178] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3832.537151] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3832.539426] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3832.543675] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3832.546263] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3836.058071] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3836.068670] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3836.107042] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3836.121235] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3836.127853] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3836.132076] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3836.138997] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3836.154445] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 3836.158094] RAX: 00000000000047a8 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3836.161927] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3836.165844] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3836.169737] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3836.173190] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3836.176717] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3836.180700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3836.183520] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3836.186871] Call Trace: [ 3836.189019] [ 3836.190044] ? show_regs.cold.9+0x22/0x2f [ 3836.192925] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3836.195710] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3836.199846] ? watchdog+0x40/0x40 [ 3836.201986] ? hrtimer_interrupt+0x128/0x2c0 [ 3836.204399] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3836.213390] ? apic_timer_interrupt+0xf/0x20 [ 3836.215950] [ 3836.218612] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3836.224095] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3836.227541] do_raw_spin_lock+0x177/0x190 [ 3836.230674] _raw_spin_lock+0x21/0x30 [ 3836.233457] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3836.237534] process_one_work+0x2c8/0x700 [ 3836.239963] worker_thread+0x296/0x6e0 [ 3836.242479] ? rescuer_thread+0x570/0x570 [ 3836.244998] kthread+0x1d1/0x200 [ 3836.246638] ? set_kthread_struct+0x70/0x70 [ 3836.248054] ret_from_fork+0x1f/0x30 [ 3860.053408] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3860.059050] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 3860.059982] Modules linked in: [ 3860.070662] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) [ 3860.080638] lustre(O) [ 3860.080640] osc(O) lmv(O) ec(O) fid(O) [ 3860.088800] osp(O) [ 3860.090577] fld(O) [ 3860.091973] ofd(O) [ 3860.094601] ptlrpc_gss(O) [ 3860.095363] lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3860.095435] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3860.095441] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3860.095444] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3860.095456] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3860.095460] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3860.095465] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3860.095468] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3860.095471] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3860.095474] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3860.095477] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3860.095480] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3860.095483] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3860.095485] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3860.095494] Call Trace: [ 3860.095498] [ 3860.095514] ? show_regs.cold.9+0x22/0x2f [ 3860.095524] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3860.095530] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3860.095534] ? watchdog+0x40/0x40 [ 3860.095539] ? hrtimer_interrupt+0x128/0x2c0 [ 3860.095542] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3860.095549] ? apic_timer_interrupt+0xf/0x20 [ 3860.095553] [ 3860.095556] ? native_safe_halt+0x2e/0x50 [ 3860.095560] kvm_wait+0xcd/0xe0 [ 3860.095568] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3860.095575] do_raw_spin_lock+0x177/0x190 [ 3860.095579] _raw_spin_lock+0x21/0x30 [ 3860.095584] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3860.095845] obd_mod_free+0x46/0x140 [ptlrpc] [ 3860.096048] mdc_commit_open+0x91/0xb0 [mdc] [ 3860.097320] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3860.097568] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3860.097783] after_reply+0xe2c/0x1780 [ptlrpc] [ 3860.097995] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3860.098353] ? __next_timer_interrupt+0x160/0x160 [ 3860.098363] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3860.098584] ? do_wait_intr+0xf0/0xf0 [ 3860.098593] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3860.098813] kthread+0x1d1/0x200 [ 3860.098818] ? set_kthread_struct+0x70/0x70 [ 3860.098822] ret_from_fork+0x1f/0x30 [ 3860.406490] ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3860.451311] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3860.463415] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3860.469637] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3860.472860] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3860.485811] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3860.489986] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3860.493346] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3860.497627] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3860.501357] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3860.504400] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3860.509722] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3860.520733] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3860.524025] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3860.533180] Call Trace: [ 3860.534398] [ 3860.535788] ? show_regs.cold.9+0x22/0x2f [ 3860.537464] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3860.539931] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3860.542936] ? watchdog+0x40/0x40 [ 3860.545472] ? hrtimer_interrupt+0x128/0x2c0 [ 3860.548676] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3860.551795] ? apic_timer_interrupt+0xf/0x20 [ 3860.554579] [ 3860.555679] ? native_safe_halt+0x2e/0x50 [ 3860.557970] kvm_wait+0xcd/0xe0 [ 3860.560343] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3860.563745] do_raw_spin_lock+0x177/0x190 [ 3860.566214] _raw_spin_lock+0x21/0x30 [ 3860.568426] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3860.571666] ? do_raw_spin_unlock+0x75/0x190 [ 3860.573596] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3860.576797] ? do_wait_intr+0xf0/0xf0 [ 3860.579659] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3860.584604] mdc_reint+0x2d/0x110 [mdc] [ 3860.587645] mdc_unlink+0x658/0x870 [mdc] [ 3860.589702] lmv_unlink+0x34d/0xbd0 [lmv] [ 3860.592302] ll_unlink+0x24d/0x7f0 [lustre] [ 3860.594793] vfs_unlink+0x202/0x330 [ 3860.597839] do_unlinkat+0x2fa/0x460 [ 3860.600656] __x64_sys_unlink+0x37/0x50 [ 3860.603353] do_syscall_64+0xc1/0x440 [ 3860.606404] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3860.617710] RIP: 0033:0x7f33957fc10b [ 3860.619249] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3860.632433] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3860.637708] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3860.641983] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3860.646043] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3860.650402] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3860.654387] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3864.058106] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3864.061659] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3864.096186] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3864.102476] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3864.108160] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3864.114356] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3864.118486] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3864.134376] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3864.144929] RAX: 0000000000005eba RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3864.150333] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3864.157915] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3864.161321] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3864.166502] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3864.174602] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3864.181415] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3864.184538] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3864.193810] Call Trace: [ 3864.194580] [ 3864.195479] ? show_regs.cold.9+0x22/0x2f [ 3864.197958] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3864.199629] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3864.204486] ? watchdog+0x40/0x40 [ 3864.205359] ? hrtimer_interrupt+0x128/0x2c0 [ 3864.208531] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3864.212057] ? apic_timer_interrupt+0xf/0x20 [ 3864.213235] [ 3864.214148] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3864.216871] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3864.219642] do_raw_spin_lock+0x177/0x190 [ 3864.221536] _raw_spin_lock+0x21/0x30 [ 3864.223198] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3864.225453] process_one_work+0x2c8/0x700 [ 3864.227157] worker_thread+0x296/0x6e0 [ 3864.228092] ? rescuer_thread+0x570/0x570 [ 3864.229291] kthread+0x1d1/0x200 [ 3864.231669] ? set_kthread_struct+0x70/0x70 [ 3864.234494] ret_from_fork+0x1f/0x30 [ 3888.053395] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3888.059124] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3888.059267] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3888.106566] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3888.106581] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3888.106584] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3888.106600] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3888.106604] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3888.106611] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3888.106613] RDX: 0000000000007d7c RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3888.106616] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3888.106618] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3888.106621] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3888.106624] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3888.106627] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3888.106630] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3888.106638] Call Trace: [ 3888.106644] [ 3888.106649] ? show_regs.cold.9+0x22/0x2f [ 3888.106658] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3888.106665] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3888.106669] ? watchdog+0x40/0x40 [ 3888.106674] ? hrtimer_interrupt+0x128/0x2c0 [ 3888.106678] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3888.106687] ? apic_timer_interrupt+0xf/0x20 [ 3888.106691] [ 3888.106694] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3888.106699] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3888.106704] do_raw_spin_lock+0x177/0x190 [ 3888.106708] _raw_spin_lock+0x21/0x30 [ 3888.106714] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3888.107707] obd_mod_free+0x46/0x140 [ptlrpc] [ 3888.114297] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3888.114376] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3888.114383] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3888.114386] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3888.114400] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3888.114404] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3888.114409] RAX: 0000000000001f62 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3888.114411] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3888.114413] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3888.114414] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3888.114416] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3888.114419] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3888.114421] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3888.114423] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3888.114432] Call Trace: [ 3888.114437] [ 3888.114443] ? show_regs.cold.9+0x22/0x2f [ 3888.114452] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3888.114459] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3888.114463] ? watchdog+0x40/0x40 [ 3888.114467] ? hrtimer_interrupt+0x128/0x2c0 [ 3888.114471] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3888.114479] ? apic_timer_interrupt+0xf/0x20 [ 3888.114483] [ 3888.114486] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3888.114490] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3888.114495] do_raw_spin_lock+0x177/0x190 [ 3888.114500] _raw_spin_lock+0x21/0x30 [ 3888.114505] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3888.114780] ? do_raw_spin_unlock+0x75/0x190 [ 3888.114786] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3888.114993] ? do_wait_intr+0xf0/0xf0 [ 3888.115002] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3888.115682] mdc_reint+0x2d/0x110 [mdc] [ 3888.115723] mdc_unlink+0x658/0x870 [mdc] [ 3888.115751] lmv_unlink+0x34d/0xbd0 [lmv] [ 3888.115799] ll_unlink+0x24d/0x7f0 [lustre] [ 3888.116443] vfs_unlink+0x202/0x330 [ 3888.116452] do_unlinkat+0x2fa/0x460 [ 3888.116458] __x64_sys_unlink+0x37/0x50 [ 3888.116461] do_syscall_64+0xc1/0x440 [ 3888.116468] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3888.116476] RIP: 0033:0x7f33957fc10b [ 3888.116483] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3888.116487] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3888.116493] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3888.116496] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3888.116498] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3888.116500] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3888.116502] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3888.558916] mdc_commit_open+0x91/0xb0 [mdc] [ 3888.562060] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3888.564803] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3888.572361] after_reply+0xe2c/0x1780 [ptlrpc] [ 3888.575926] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3888.580958] ? __next_timer_interrupt+0x160/0x160 [ 3888.584261] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3888.586928] ? do_wait_intr+0xf0/0xf0 [ 3888.593240] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3888.597800] kthread+0x1d1/0x200 [ 3888.600638] ? set_kthread_struct+0x70/0x70 [ 3888.606640] ret_from_fork+0x1f/0x30 [ 3892.058380] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3892.064659] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3892.128704] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3892.140848] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3892.147265] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3892.156804] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3892.160422] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3892.178001] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3892.183347] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3892.187610] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3892.192104] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3892.197545] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3892.206585] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3892.212378] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3892.222027] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3892.227811] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3892.235521] Call Trace: [ 3892.237514] [ 3892.241199] ? show_regs.cold.9+0x22/0x2f [ 3892.243764] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3892.249941] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3892.252075] ? watchdog+0x40/0x40 [ 3892.254115] ? hrtimer_interrupt+0x128/0x2c0 [ 3892.256017] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3892.258345] ? apic_timer_interrupt+0xf/0x20 [ 3892.260409] [ 3892.261604] ? native_safe_halt+0x2e/0x50 [ 3892.263938] kvm_wait+0xcd/0xe0 [ 3892.265340] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3892.267837] do_raw_spin_lock+0x177/0x190 [ 3892.270191] _raw_spin_lock+0x21/0x30 [ 3892.281962] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3892.285261] process_one_work+0x2c8/0x700 [ 3892.290295] worker_thread+0x296/0x6e0 [ 3892.293281] ? rescuer_thread+0x570/0x570 [ 3892.298057] kthread+0x1d1/0x200 [ 3892.300564] ? set_kthread_struct+0x70/0x70 [ 3892.302524] ret_from_fork+0x1f/0x30 [ 3916.054649] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3916.064876] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3916.065160] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3916.104502] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3916.104731] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3916.104735] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3916.104749] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 3916.104753] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3916.104759] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3916.104761] RDX: 0000000000000286 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3916.104763] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3916.104765] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3916.104767] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3916.104771] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3916.104773] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3916.104776] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3916.104783] Call Trace: [ 3916.104789] [ 3916.104795] ? show_regs.cold.9+0x22/0x2f [ 3916.104806] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3916.104813] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3916.104818] ? watchdog+0x40/0x40 [ 3916.104822] ? hrtimer_interrupt+0x128/0x2c0 [ 3916.104826] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3916.104833] ? apic_timer_interrupt+0xf/0x20 [ 3916.104838] [ 3916.104840] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 3916.104845] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3916.104849] do_raw_spin_lock+0x177/0x190 [ 3916.104854] _raw_spin_lock+0x21/0x30 [ 3916.104859] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3916.114254] Modules linked in: [ 3916.121265] obd_mod_free+0x46/0x140 [ptlrpc] [ 3916.121572] mdc_commit_open+0x91/0xb0 [mdc] [ 3916.121615] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3916.121810] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3916.122192] after_reply+0xe2c/0x1780 [ptlrpc] [ 3916.122387] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3916.122591] ? __next_timer_interrupt+0x160/0x160 [ 3916.122603] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3916.122805] ? do_wait_intr+0xf0/0xf0 [ 3916.122814] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3916.123257] kthread+0x1d1/0x200 [ 3916.123264] ? set_kthread_struct+0x70/0x70 [ 3916.123267] ret_from_fork+0x1f/0x30 [ 3916.304805] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3916.336650] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3916.341474] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3916.345623] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3916.348327] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3916.355910] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3916.358904] RAX: 000000000000788a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3916.361856] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3916.364794] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3916.367543] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3916.370261] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3916.372929] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3916.376029] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3916.378456] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3916.381275] Call Trace: [ 3916.382194] [ 3916.382895] ? show_regs.cold.9+0x22/0x2f [ 3916.384906] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3916.387324] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3916.389565] ? watchdog+0x40/0x40 [ 3916.390816] ? hrtimer_interrupt+0x128/0x2c0 [ 3916.394619] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3916.396326] ? apic_timer_interrupt+0xf/0x20 [ 3916.397698] [ 3916.398620] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3916.401362] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3916.404077] do_raw_spin_lock+0x177/0x190 [ 3916.407333] _raw_spin_lock+0x21/0x30 [ 3916.409023] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3916.411643] ? do_raw_spin_unlock+0x75/0x190 [ 3916.413700] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3916.416128] ? do_wait_intr+0xf0/0xf0 [ 3916.418503] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3916.421242] mdc_reint+0x2d/0x110 [mdc] [ 3916.422943] mdc_unlink+0x658/0x870 [mdc] [ 3916.425193] lmv_unlink+0x34d/0xbd0 [lmv] [ 3916.427824] ll_unlink+0x24d/0x7f0 [lustre] [ 3916.429986] vfs_unlink+0x202/0x330 [ 3916.431775] do_unlinkat+0x2fa/0x460 [ 3916.433727] __x64_sys_unlink+0x37/0x50 [ 3916.436213] do_syscall_64+0xc1/0x440 [ 3916.437905] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3916.440101] RIP: 0033:0x7f33957fc10b [ 3916.441512] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3916.450679] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3916.454994] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3916.459543] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3916.463510] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3916.466206] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3916.468426] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3920.058112] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3920.060640] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3920.090231] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3920.099848] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3920.109695] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3920.114270] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3920.117548] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3920.139218] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3920.146811] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3920.163519] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3920.165277] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3920.177663] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3920.182943] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3920.185342] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3920.193280] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3920.196797] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3920.205801] Call Trace: [ 3920.209215] [ 3920.213474] ? show_regs.cold.9+0x22/0x2f [ 3920.214885] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3920.217333] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3920.223826] ? watchdog+0x40/0x40 [ 3920.225808] ? hrtimer_interrupt+0x128/0x2c0 [ 3920.230439] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3920.233589] ? apic_timer_interrupt+0xf/0x20 [ 3920.242003] [ 3920.243627] ? native_safe_halt+0x2e/0x50 [ 3920.246029] kvm_wait+0xcd/0xe0 [ 3920.247448] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3920.251802] do_raw_spin_lock+0x177/0x190 [ 3920.254425] _raw_spin_lock+0x21/0x30 [ 3920.255763] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3920.264657] process_one_work+0x2c8/0x700 [ 3920.267126] worker_thread+0x296/0x6e0 [ 3920.270437] ? rescuer_thread+0x570/0x570 [ 3920.274512] kthread+0x1d1/0x200 [ 3920.278860] ? set_kthread_struct+0x70/0x70 [ 3920.283627] ret_from_fork+0x1f/0x30 [ 3944.053057] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 3944.059026] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3944.061852] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3944.061932] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3944.061939] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3944.061942] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3944.061957] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3944.061961] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 3944.061965] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3944.061967] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3944.061969] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3944.061971] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3944.061973] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3944.061975] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3944.061978] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3944.061980] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3944.061988] Call Trace: [ 3944.061992] [ 3944.062041] ? show_regs.cold.9+0x22/0x2f [ 3944.062053] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3944.062059] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3944.062180] ? watchdog+0x40/0x40 [ 3944.062187] ? hrtimer_interrupt+0x128/0x2c0 [ 3944.062191] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3944.062197] ? apic_timer_interrupt+0xf/0x20 [ 3944.062201] [ 3944.062204] ? native_safe_halt+0x2e/0x50 [ 3944.062208] kvm_wait+0xcd/0xe0 [ 3944.062215] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3944.062223] do_raw_spin_lock+0x177/0x190 [ 3944.062227] _raw_spin_lock+0x21/0x30 [ 3944.062232] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3944.065606] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3944.065680] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3944.065687] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3944.065690] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3944.065702] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3944.065707] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3944.065711] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3944.065713] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3944.065715] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3944.065717] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3944.065719] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3944.065722] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3944.065725] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3944.065728] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3944.065735] Call Trace: [ 3944.065739] [ 3944.065744] ? show_regs.cold.9+0x22/0x2f [ 3944.065754] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3944.065760] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3944.065764] ? watchdog+0x40/0x40 [ 3944.065768] ? hrtimer_interrupt+0x128/0x2c0 [ 3944.065773] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3944.065780] ? apic_timer_interrupt+0xf/0x20 [ 3944.065785] [ 3944.065788] ? native_safe_halt+0x2e/0x50 [ 3944.065792] kvm_wait+0xcd/0xe0 [ 3944.065800] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3944.065808] do_raw_spin_lock+0x177/0x190 [ 3944.065811] _raw_spin_lock+0x21/0x30 [ 3944.065816] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3944.067018] ? do_raw_spin_unlock+0x75/0x190 [ 3944.067025] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3944.067247] ? do_wait_intr+0xf0/0xf0 [ 3944.067255] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3944.131450] obd_mod_free+0x46/0x140 [ptlrpc] [ 3944.131710] mdc_commit_open+0x91/0xb0 [mdc] [ 3944.131743] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3944.150896] mdc_reint+0x2d/0x110 [mdc] [ 3944.150943] mdc_unlink+0x658/0x870 [mdc] [ 3944.150970] lmv_unlink+0x34d/0xbd0 [lmv] [ 3944.151002] ll_unlink+0x24d/0x7f0 [lustre] [ 3944.151117] vfs_unlink+0x202/0x330 [ 3944.151128] do_unlinkat+0x2fa/0x460 [ 3944.151133] __x64_sys_unlink+0x37/0x50 [ 3944.151137] do_syscall_64+0xc1/0x440 [ 3944.151143] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3944.151153] RIP: 0033:0x7f33957fc10b [ 3944.151160] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3944.151164] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3944.151170] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3944.151172] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3944.151174] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3944.151176] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3944.151178] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3944.674143] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3944.683192] after_reply+0xe2c/0x1780 [ptlrpc] [ 3944.689708] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3944.699169] ? __next_timer_interrupt+0x160/0x160 [ 3944.702581] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3944.705358] ? do_wait_intr+0xf0/0xf0 [ 3944.710912] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3944.722146] kthread+0x1d1/0x200 [ 3944.723627] ? set_kthread_struct+0x70/0x70 [ 3944.728712] ret_from_fork+0x1f/0x30 [ 3948.062859] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3948.076890] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3948.169999] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3948.186948] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3948.192834] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3948.198105] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3948.209410] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3948.233913] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3948.239514] RAX: 0000000000007ca8 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3948.244661] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3948.253792] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3948.267513] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3948.275152] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3948.288559] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3948.297722] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3948.305795] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3948.317015] Call Trace: [ 3948.318963] [ 3948.319764] ? show_regs.cold.9+0x22/0x2f [ 3948.321187] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3948.324613] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3948.331271] ? watchdog+0x40/0x40 [ 3948.332397] ? hrtimer_interrupt+0x128/0x2c0 [ 3948.338324] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3948.343780] ? apic_timer_interrupt+0xf/0x20 [ 3948.351911] [ 3948.355381] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3948.362012] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3948.372469] do_raw_spin_lock+0x177/0x190 [ 3948.377560] _raw_spin_lock+0x21/0x30 [ 3948.380647] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3948.387841] process_one_work+0x2c8/0x700 [ 3948.390548] worker_thread+0x296/0x6e0 [ 3948.393062] ? rescuer_thread+0x570/0x570 [ 3948.396582] kthread+0x1d1/0x200 [ 3948.398283] ? set_kthread_struct+0x70/0x70 [ 3948.401774] ret_from_fork+0x1f/0x30 [ 3958.064013] rcu: INFO: rcu_sched self-detected stall on CPU [ 3958.068237] rcu: 0-....: (1159198 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=350396 [ 3958.079324] (t=1860044 jiffies g=297885 q=10104) [ 3958.086367] NMI backtrace for cpu 0 [ 3958.090988] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3958.100763] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3958.105568] Call Trace: [ 3958.106892] [ 3958.111051] ? dump_stack+0xbb/0x10e [ 3958.115766] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 3958.118819] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 3958.129201] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 3958.133882] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 3958.138949] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 3958.143131] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 3958.146885] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 3958.150452] ? update_process_times+0x6d/0xc0 [ 3958.153067] ? tick_sched_handle.isra.18+0x27/0x90 [ 3958.156174] ? tick_sched_timer+0x53/0xb0 [ 3958.161700] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3958.164750] ? tick_sched_handle.isra.18+0x90/0x90 [ 3958.167836] ? hrtimer_interrupt+0x128/0x2c0 [ 3958.170565] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3958.173277] ? apic_timer_interrupt+0xf/0x20 [ 3958.175370] [ 3958.176627] ? __pv_queued_spin_lock_slowpath+0x17c/0x6d0 [ 3958.179282] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3958.182340] ? do_raw_spin_lock+0x177/0x190 [ 3958.187872] ? _raw_spin_lock+0x21/0x30 [ 3958.191578] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3958.197804] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 3958.202051] ? mdc_commit_open+0x91/0xb0 [mdc] [ 3958.205663] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3958.209090] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3958.213235] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 3958.226574] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3958.228725] ? __next_timer_interrupt+0x160/0x160 [ 3958.242212] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3958.251055] ? do_wait_intr+0xf0/0xf0 [ 3958.258313] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3958.264910] ? kthread+0x1d1/0x200 [ 3958.270493] ? set_kthread_struct+0x70/0x70 [ 3958.274053] ? ret_from_fork+0x1f/0x30 [ 3972.059060] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 3972.064293] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3972.106654] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3972.111361] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3972.115068] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3972.116820] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3972.125755] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 3972.131626] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3972.140801] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 3972.146896] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 3972.152918] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 3972.157443] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 3972.166235] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 3972.171737] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3972.176263] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 3972.186374] Call Trace: [ 3972.187195] [ 3972.188570] ? show_regs.cold.9+0x22/0x2f [ 3972.193314] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3972.195785] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3972.198918] ? watchdog+0x40/0x40 [ 3972.203965] ? hrtimer_interrupt+0x128/0x2c0 [ 3972.206798] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3972.210792] ? apic_timer_interrupt+0xf/0x20 [ 3972.216965] [ 3972.218733] ? native_safe_halt+0x2e/0x50 [ 3972.221600] kvm_wait+0xcd/0xe0 [ 3972.226247] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3972.234451] do_raw_spin_lock+0x177/0x190 [ 3972.238803] _raw_spin_lock+0x21/0x30 [ 3972.243157] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 3972.249192] ? do_raw_spin_unlock+0x75/0x190 [ 3972.255922] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 3972.267655] ? do_wait_intr+0xf0/0xf0 [ 3972.269311] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 3972.273146] mdc_reint+0x2d/0x110 [mdc] [ 3972.278194] mdc_unlink+0x658/0x870 [mdc] [ 3972.280948] lmv_unlink+0x34d/0xbd0 [lmv] [ 3972.283162] ll_unlink+0x24d/0x7f0 [lustre] [ 3972.286347] vfs_unlink+0x202/0x330 [ 3972.290075] do_unlinkat+0x2fa/0x460 [ 3972.294221] __x64_sys_unlink+0x37/0x50 [ 3972.297708] do_syscall_64+0xc1/0x440 [ 3972.299954] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 3972.303180] RIP: 0033:0x7f33957fc10b [ 3972.309014] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 3972.321756] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 3972.327298] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 3972.334443] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 3972.338191] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 3972.346297] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 3972.357392] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 3976.058324] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 3976.061448] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3976.080038] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3976.084274] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3976.087688] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 3976.090892] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3976.093516] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 3976.100692] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3976.103238] RAX: 0000000000000509 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 3976.105492] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 3976.107938] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 3976.111232] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 3976.114463] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 3976.127950] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 3976.135376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3976.142534] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 3976.149914] Call Trace: [ 3976.151254] [ 3976.152450] ? show_regs.cold.9+0x22/0x2f [ 3976.156591] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3976.162235] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3976.165270] ? watchdog+0x40/0x40 [ 3976.167818] ? hrtimer_interrupt+0x128/0x2c0 [ 3976.170166] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3976.171606] ? apic_timer_interrupt+0xf/0x20 [ 3976.174334] [ 3976.175398] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 3976.177373] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 3976.179637] do_raw_spin_lock+0x177/0x190 [ 3976.181431] _raw_spin_lock+0x21/0x30 [ 3976.183966] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 3976.187528] process_one_work+0x2c8/0x700 [ 3976.189366] worker_thread+0x296/0x6e0 [ 3976.190940] ? rescuer_thread+0x570/0x570 [ 3976.192603] kthread+0x1d1/0x200 [ 3976.194122] ? set_kthread_struct+0x70/0x70 [ 3976.195495] ret_from_fork+0x1f/0x30 [ 3984.053038] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 3984.055996] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 3984.072138] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 3984.075238] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 3984.077709] RIP: 0010:native_safe_halt+0x2e/0x50 [ 3984.079127] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 3984.084972] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 3984.087499] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 3984.089095] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 3984.090992] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 3984.093057] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 3984.095033] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 3984.097461] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 3984.099352] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3984.101073] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 3984.103271] Call Trace: [ 3984.103988] [ 3984.104535] ? show_regs.cold.9+0x22/0x2f [ 3984.105796] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 3984.107113] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 3984.108536] ? watchdog+0x40/0x40 [ 3984.109478] ? hrtimer_interrupt+0x128/0x2c0 [ 3984.110741] ? smp_apic_timer_interrupt+0xa6/0x270 [ 3984.111961] ? apic_timer_interrupt+0xf/0x20 [ 3984.112959] [ 3984.113442] ? native_safe_halt+0x2e/0x50 [ 3984.114350] kvm_wait+0xcd/0xe0 [ 3984.115854] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 3984.118313] do_raw_spin_lock+0x177/0x190 [ 3984.120114] _raw_spin_lock+0x21/0x30 [ 3984.121941] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 3984.123784] obd_mod_free+0x46/0x140 [ptlrpc] [ 3984.126710] mdc_commit_open+0x91/0xb0 [mdc] [ 3984.130538] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 3984.132283] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 3984.139846] after_reply+0xe2c/0x1780 [ptlrpc] [ 3984.142844] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 3984.146360] ? __next_timer_interrupt+0x160/0x160 [ 3984.149667] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 3984.151893] ? do_wait_intr+0xf0/0xf0 [ 3984.153835] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 3984.156491] kthread+0x1d1/0x200 [ 3984.158184] ? set_kthread_struct+0x70/0x70 [ 3984.159999] ret_from_fork+0x1f/0x30 [ 4000.059152] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4000.072187] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4000.126975] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4000.142077] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4000.155302] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4000.162784] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4000.185944] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4000.199990] RAX: 0000000000006318 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4000.207267] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4000.215984] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4000.223321] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4000.232996] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4000.240887] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4000.250728] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4000.259198] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4000.270614] Call Trace: [ 4000.278638] [ 4000.281403] ? show_regs.cold.9+0x22/0x2f [ 4000.287868] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4000.294476] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4000.297143] ? watchdog+0x40/0x40 [ 4000.298596] ? hrtimer_interrupt+0x128/0x2c0 [ 4000.305736] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4000.311565] ? apic_timer_interrupt+0xf/0x20 [ 4000.314462] [ 4000.318499] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4000.322885] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4000.328502] do_raw_spin_lock+0x177/0x190 [ 4000.333112] _raw_spin_lock+0x21/0x30 [ 4000.336657] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4000.342345] ? do_raw_spin_unlock+0x75/0x190 [ 4000.344545] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4000.346805] ? do_wait_intr+0xf0/0xf0 [ 4000.352422] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4000.356024] mdc_reint+0x2d/0x110 [mdc] [ 4000.357817] mdc_unlink+0x658/0x870 [mdc] [ 4000.361830] lmv_unlink+0x34d/0xbd0 [lmv] [ 4000.366451] ll_unlink+0x24d/0x7f0 [lustre] [ 4000.369954] vfs_unlink+0x202/0x330 [ 4000.375140] do_unlinkat+0x2fa/0x460 [ 4000.380621] __x64_sys_unlink+0x37/0x50 [ 4000.384927] do_syscall_64+0xc1/0x440 [ 4000.388627] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4000.397512] RIP: 0033:0x7f33957fc10b [ 4000.400871] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4000.417685] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4000.423076] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4000.429159] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4000.434120] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4000.441760] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4000.449154] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4004.059345] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4004.064178] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4004.085037] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4004.090115] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4004.093904] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4004.097007] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4004.100122] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4004.109890] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4004.114109] RAX: 0000000000004286 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4004.117707] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4004.121063] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4004.124428] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4004.128932] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4004.140463] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4004.154768] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4004.158272] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4004.167435] Call Trace: [ 4004.169463] [ 4004.171152] ? show_regs.cold.9+0x22/0x2f [ 4004.175843] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4004.182409] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4004.184022] ? watchdog+0x40/0x40 [ 4004.186026] ? hrtimer_interrupt+0x128/0x2c0 [ 4004.187968] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4004.194935] ? apic_timer_interrupt+0xf/0x20 [ 4004.197964] [ 4004.200555] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4004.205352] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4004.210041] do_raw_spin_lock+0x177/0x190 [ 4004.215054] _raw_spin_lock+0x21/0x30 [ 4004.218023] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4004.225563] process_one_work+0x2c8/0x700 [ 4004.228845] worker_thread+0x296/0x6e0 [ 4004.232845] ? rescuer_thread+0x570/0x570 [ 4004.237839] kthread+0x1d1/0x200 [ 4004.240262] ? set_kthread_struct+0x70/0x70 [ 4004.245272] ret_from_fork+0x1f/0x30 [ 4012.053574] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4012.059484] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4012.174007] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4012.192246] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4012.203442] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4012.212868] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4012.227192] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4012.229749] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4012.232115] RDX: 0000000000000bc2 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4012.236669] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4012.241253] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4012.243983] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4012.246834] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4012.249705] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4012.253530] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4012.255983] Call Trace: [ 4012.256795] [ 4012.257536] ? show_regs.cold.9+0x22/0x2f [ 4012.258788] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4012.260473] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4012.267384] ? watchdog+0x40/0x40 [ 4012.271100] ? hrtimer_interrupt+0x128/0x2c0 [ 4012.277216] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4012.281471] ? apic_timer_interrupt+0xf/0x20 [ 4012.289267] [ 4012.295319] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4012.296834] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4012.308384] do_raw_spin_lock+0x177/0x190 [ 4012.310830] _raw_spin_lock+0x21/0x30 [ 4012.316562] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4012.319591] obd_mod_free+0x46/0x140 [ptlrpc] [ 4012.322905] mdc_commit_open+0x91/0xb0 [mdc] [ 4012.324788] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4012.327518] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4012.330099] after_reply+0xe2c/0x1780 [ptlrpc] [ 4012.332642] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4012.335682] ? __next_timer_interrupt+0x160/0x160 [ 4012.337569] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4012.340817] ? do_wait_intr+0xf0/0xf0 [ 4012.343623] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4012.346593] kthread+0x1d1/0x200 [ 4012.347761] ? set_kthread_struct+0x70/0x70 [ 4012.351181] ret_from_fork+0x1f/0x30 [ 4028.059020] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4028.062237] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4028.125653] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4028.138394] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4028.160161] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4028.167130] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4028.195106] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4028.204297] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4028.214961] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4028.240132] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4028.252619] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4028.266927] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4028.283751] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4028.300327] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4028.309030] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4028.321113] Call Trace: [ 4028.325069] [ 4028.328468] ? show_regs.cold.9+0x22/0x2f [ 4028.333675] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4028.338216] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4028.346920] ? watchdog+0x40/0x40 [ 4028.352520] ? hrtimer_interrupt+0x128/0x2c0 [ 4028.356916] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4028.364293] ? apic_timer_interrupt+0xf/0x20 [ 4028.371159] [ 4028.391609] ? native_safe_halt+0x2e/0x50 [ 4028.395897] kvm_wait+0xcd/0xe0 [ 4028.404711] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4028.413728] do_raw_spin_lock+0x177/0x190 [ 4028.418535] _raw_spin_lock+0x21/0x30 [ 4028.426012] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4028.436924] ? do_raw_spin_unlock+0x75/0x190 [ 4028.446097] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4028.451617] ? do_wait_intr+0xf0/0xf0 [ 4028.453346] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4028.456127] mdc_reint+0x2d/0x110 [mdc] [ 4028.457514] mdc_unlink+0x658/0x870 [mdc] [ 4028.458945] lmv_unlink+0x34d/0xbd0 [lmv] [ 4028.461827] ll_unlink+0x24d/0x7f0 [lustre] [ 4028.470690] vfs_unlink+0x202/0x330 [ 4028.473344] do_unlinkat+0x2fa/0x460 [ 4028.475857] __x64_sys_unlink+0x37/0x50 [ 4028.480495] do_syscall_64+0xc1/0x440 [ 4028.481768] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4028.493378] RIP: 0033:0x7f33957fc10b [ 4028.506197] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4028.530113] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4028.541173] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4028.554699] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4028.564671] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4028.571560] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4028.582644] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4032.059507] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4032.067036] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4032.111696] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4032.126687] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4032.134892] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4032.141273] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4032.146919] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4032.159062] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4032.166174] RAX: 0000000000007d95 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4032.170918] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4032.174763] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4032.182408] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4032.189918] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4032.199337] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4032.202829] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4032.206172] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4032.211508] Call Trace: [ 4032.212583] [ 4032.213997] ? show_regs.cold.9+0x22/0x2f [ 4032.215769] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4032.222624] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4032.226885] ? watchdog+0x40/0x40 [ 4032.235593] ? hrtimer_interrupt+0x128/0x2c0 [ 4032.242481] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4032.244581] ? apic_timer_interrupt+0xf/0x20 [ 4032.249542] [ 4032.252637] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4032.259290] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4032.263640] do_raw_spin_lock+0x177/0x190 [ 4032.270875] _raw_spin_lock+0x21/0x30 [ 4032.276710] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4032.282113] process_one_work+0x2c8/0x700 [ 4032.284423] worker_thread+0x296/0x6e0 [ 4032.286801] ? rescuer_thread+0x570/0x570 [ 4032.289395] kthread+0x1d1/0x200 [ 4032.291510] ? set_kthread_struct+0x70/0x70 [ 4032.293810] ret_from_fork+0x1f/0x30 [ 4040.053063] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4040.058413] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4040.083808] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4040.089057] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4040.093119] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4040.095290] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4040.103887] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4040.107603] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4040.110849] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4040.113622] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4040.116488] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4040.119396] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4040.123937] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4040.127276] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4040.129970] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4040.133287] Call Trace: [ 4040.134594] [ 4040.135432] ? show_regs.cold.9+0x22/0x2f [ 4040.137072] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4040.138910] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4040.140789] ? watchdog+0x40/0x40 [ 4040.142201] ? hrtimer_interrupt+0x128/0x2c0 [ 4040.143958] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4040.145871] ? apic_timer_interrupt+0xf/0x20 [ 4040.147587] [ 4040.148398] ? native_safe_halt+0x2e/0x50 [ 4040.150144] kvm_wait+0xcd/0xe0 [ 4040.151361] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4040.153614] do_raw_spin_lock+0x177/0x190 [ 4040.155242] _raw_spin_lock+0x21/0x30 [ 4040.156846] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4040.159139] obd_mod_free+0x46/0x140 [ptlrpc] [ 4040.161252] mdc_commit_open+0x91/0xb0 [mdc] [ 4040.163133] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4040.165344] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4040.167858] after_reply+0xe2c/0x1780 [ptlrpc] [ 4040.170110] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4040.173077] ? __next_timer_interrupt+0x160/0x160 [ 4040.175107] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4040.177208] ? do_wait_intr+0xf0/0xf0 [ 4040.178748] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4040.181370] kthread+0x1d1/0x200 [ 4040.182738] ? set_kthread_struct+0x70/0x70 [ 4040.184854] ret_from_fork+0x1f/0x30 [ 4056.059290] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4056.071345] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4056.116127] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4056.128795] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4056.138470] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4056.144996] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4056.161322] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4056.165399] RAX: 000000000000301a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4056.168623] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4056.171744] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4056.175658] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4056.179279] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4056.182723] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4056.186548] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4056.191243] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4056.194499] Call Trace: [ 4056.195344] [ 4056.203723] ? show_regs.cold.9+0x22/0x2f [ 4056.207881] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4056.213621] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4056.218085] ? watchdog+0x40/0x40 [ 4056.219757] ? hrtimer_interrupt+0x128/0x2c0 [ 4056.223186] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4056.227778] ? apic_timer_interrupt+0xf/0x20 [ 4056.232470] [ 4056.237328] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4056.240162] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4056.242519] do_raw_spin_lock+0x177/0x190 [ 4056.245447] _raw_spin_lock+0x21/0x30 [ 4056.249637] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4056.252618] ? do_raw_spin_unlock+0x75/0x190 [ 4056.259638] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4056.265089] ? do_wait_intr+0xf0/0xf0 [ 4056.267668] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4056.273951] mdc_reint+0x2d/0x110 [mdc] [ 4056.278387] mdc_unlink+0x658/0x870 [mdc] [ 4056.281757] lmv_unlink+0x34d/0xbd0 [lmv] [ 4056.285676] ll_unlink+0x24d/0x7f0 [lustre] [ 4056.290924] vfs_unlink+0x202/0x330 [ 4056.296555] do_unlinkat+0x2fa/0x460 [ 4056.297879] __x64_sys_unlink+0x37/0x50 [ 4056.303875] do_syscall_64+0xc1/0x440 [ 4056.305329] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4056.310448] RIP: 0033:0x7f33957fc10b [ 4056.311830] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4056.335172] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4056.342735] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4056.348157] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4056.351697] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4056.355127] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4056.359439] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4060.058026] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4060.061308] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4060.107578] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4060.115685] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4060.119634] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4060.128745] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4060.133147] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4060.149222] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4060.151834] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4060.156175] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4060.161745] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4060.173023] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4060.178289] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4060.185813] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4060.200864] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4060.208506] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4060.211714] Call Trace: [ 4060.213968] [ 4060.215419] ? show_regs.cold.9+0x22/0x2f [ 4060.218595] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4060.224568] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4060.230474] ? watchdog+0x40/0x40 [ 4060.237556] ? hrtimer_interrupt+0x128/0x2c0 [ 4060.243284] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4060.257500] ? apic_timer_interrupt+0xf/0x20 [ 4060.263447] [ 4060.270404] ? native_safe_halt+0x2e/0x50 [ 4060.275464] kvm_wait+0xcd/0xe0 [ 4060.281908] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4060.283674] do_raw_spin_lock+0x177/0x190 [ 4060.291896] _raw_spin_lock+0x21/0x30 [ 4060.299245] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4060.305497] process_one_work+0x2c8/0x700 [ 4060.310253] worker_thread+0x296/0x6e0 [ 4060.311523] ? rescuer_thread+0x570/0x570 [ 4060.324735] kthread+0x1d1/0x200 [ 4060.330720] ? set_kthread_struct+0x70/0x70 [ 4060.334309] ret_from_fork+0x1f/0x30 [ 4068.055783] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4068.062880] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4068.129852] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4068.139399] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4068.143845] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4068.150252] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4068.165472] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4068.172827] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4068.180795] RDX: 00000000000043c2 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4068.196639] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4068.201412] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4068.205239] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4068.208485] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4068.215493] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4068.220805] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4068.230881] Call Trace: [ 4068.231750] [ 4068.232475] ? show_regs.cold.9+0x22/0x2f [ 4068.235169] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4068.239328] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4068.243879] ? watchdog+0x40/0x40 [ 4068.247637] ? hrtimer_interrupt+0x128/0x2c0 [ 4068.250914] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4068.261747] ? apic_timer_interrupt+0xf/0x20 [ 4068.265697] [ 4068.266602] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4068.275282] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4068.279795] do_raw_spin_lock+0x177/0x190 [ 4068.285082] _raw_spin_lock+0x21/0x30 [ 4068.289219] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4068.293525] obd_mod_free+0x46/0x140 [ptlrpc] [ 4068.296493] mdc_commit_open+0x91/0xb0 [mdc] [ 4068.298788] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4068.302222] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4068.305554] after_reply+0xe2c/0x1780 [ptlrpc] [ 4068.308197] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4068.310849] ? __next_timer_interrupt+0x160/0x160 [ 4068.313109] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4068.315356] ? do_wait_intr+0xf0/0xf0 [ 4068.316863] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4068.319581] kthread+0x1d1/0x200 [ 4068.320927] ? set_kthread_struct+0x70/0x70 [ 4068.322443] ret_from_fork+0x1f/0x30 [ 4084.059067] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4084.062824] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4084.099318] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4084.104740] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4084.108354] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4084.110145] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4084.121091] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4084.124794] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4084.130087] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4084.133441] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4084.136111] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4084.139608] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4084.143755] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4084.147416] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4084.152394] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4084.156129] Call Trace: [ 4084.157198] [ 4084.159937] ? show_regs.cold.9+0x22/0x2f [ 4084.162416] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4084.164990] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4084.167924] ? watchdog+0x40/0x40 [ 4084.169582] ? hrtimer_interrupt+0x128/0x2c0 [ 4084.171895] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4084.174070] ? apic_timer_interrupt+0xf/0x20 [ 4084.175746] [ 4084.176611] ? native_safe_halt+0x2e/0x50 [ 4084.178305] kvm_wait+0xcd/0xe0 [ 4084.179545] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4084.181570] do_raw_spin_lock+0x177/0x190 [ 4084.183155] _raw_spin_lock+0x21/0x30 [ 4084.184519] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4084.187064] ? do_raw_spin_unlock+0x75/0x190 [ 4084.188673] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4084.191023] ? do_wait_intr+0xf0/0xf0 [ 4084.192873] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4084.195380] mdc_reint+0x2d/0x110 [mdc] [ 4084.197043] mdc_unlink+0x658/0x870 [mdc] [ 4084.198786] lmv_unlink+0x34d/0xbd0 [lmv] [ 4084.200808] ll_unlink+0x24d/0x7f0 [lustre] [ 4084.202480] vfs_unlink+0x202/0x330 [ 4084.204017] do_unlinkat+0x2fa/0x460 [ 4084.205243] __x64_sys_unlink+0x37/0x50 [ 4084.206943] do_syscall_64+0xc1/0x440 [ 4084.208846] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4084.210876] RIP: 0033:0x7f33957fc10b [ 4084.212279] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4084.220102] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4084.223317] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4084.227158] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4084.230044] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4084.233116] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4084.236543] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4088.062284] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4088.067209] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4088.096801] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4088.104592] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4088.110895] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4088.115120] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4088.118381] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4088.136346] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4088.139249] RAX: 000000000000747c RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4088.143647] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4088.147926] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4088.151709] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4088.155254] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4088.159879] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4088.162831] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4088.164663] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4088.169928] Call Trace: [ 4088.173282] [ 4088.174708] ? show_regs.cold.9+0x22/0x2f [ 4088.176879] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4088.179863] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4088.181885] ? watchdog+0x40/0x40 [ 4088.183897] ? hrtimer_interrupt+0x128/0x2c0 [ 4088.185711] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4088.189059] ? apic_timer_interrupt+0xf/0x20 [ 4088.195653] [ 4088.200213] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4088.207534] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4088.211845] do_raw_spin_lock+0x177/0x190 [ 4088.216966] _raw_spin_lock+0x21/0x30 [ 4088.218548] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4088.224676] process_one_work+0x2c8/0x700 [ 4088.227437] worker_thread+0x296/0x6e0 [ 4088.229911] ? rescuer_thread+0x570/0x570 [ 4088.236316] kthread+0x1d1/0x200 [ 4088.239274] ? set_kthread_struct+0x70/0x70 [ 4088.240620] ret_from_fork+0x1f/0x30 [ 4096.053499] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4096.058780] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4096.108748] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4096.116473] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4096.133572] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4096.142781] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4096.162801] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4096.167630] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4096.176120] RDX: 00000000000035e1 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4096.181376] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4096.191325] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4096.204008] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4096.216162] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4096.228262] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4096.233502] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4096.240160] Call Trace: [ 4096.241057] [ 4096.241735] ? show_regs.cold.9+0x22/0x2f [ 4096.244469] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4096.249647] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4096.256084] ? watchdog+0x40/0x40 [ 4096.259368] ? hrtimer_interrupt+0x128/0x2c0 [ 4096.263710] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4096.267851] ? apic_timer_interrupt+0xf/0x20 [ 4096.270582] [ 4096.271944] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4096.274960] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4096.279087] do_raw_spin_lock+0x177/0x190 [ 4096.282001] _raw_spin_lock+0x21/0x30 [ 4096.283935] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4096.289597] obd_mod_free+0x46/0x140 [ptlrpc] [ 4096.293931] mdc_commit_open+0x91/0xb0 [mdc] [ 4096.297692] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4096.299873] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4096.302924] after_reply+0xe2c/0x1780 [ptlrpc] [ 4096.308130] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4096.311625] ? __next_timer_interrupt+0x160/0x160 [ 4096.315772] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4096.317574] ? do_wait_intr+0xf0/0xf0 [ 4096.324741] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4096.329776] kthread+0x1d1/0x200 [ 4096.331494] ? set_kthread_struct+0x70/0x70 [ 4096.333688] ret_from_fork+0x1f/0x30 [ 4112.059323] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4112.072925] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4112.120113] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4112.125388] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4112.131953] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4112.135394] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4112.166212] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4112.176692] RAX: 0000000000007084 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4112.181189] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4112.184947] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4112.189815] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4112.193979] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4112.200848] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4112.208120] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4112.210075] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4112.217206] Call Trace: [ 4112.218969] [ 4112.221024] ? show_regs.cold.9+0x22/0x2f [ 4112.229254] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4112.231401] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4112.234956] ? watchdog+0x40/0x40 [ 4112.238394] ? hrtimer_interrupt+0x128/0x2c0 [ 4112.242545] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4112.245676] ? apic_timer_interrupt+0xf/0x20 [ 4112.247730] [ 4112.248954] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4112.251855] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4112.254347] do_raw_spin_lock+0x177/0x190 [ 4112.256546] _raw_spin_lock+0x21/0x30 [ 4112.257871] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4112.264840] ? do_raw_spin_unlock+0x75/0x190 [ 4112.267966] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4112.271929] ? do_wait_intr+0xf0/0xf0 [ 4112.273904] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4112.282263] mdc_reint+0x2d/0x110 [mdc] [ 4112.285560] mdc_unlink+0x658/0x870 [mdc] [ 4112.287893] lmv_unlink+0x34d/0xbd0 [lmv] [ 4112.289748] ll_unlink+0x24d/0x7f0 [lustre] [ 4112.292635] vfs_unlink+0x202/0x330 [ 4112.298885] do_unlinkat+0x2fa/0x460 [ 4112.303511] __x64_sys_unlink+0x37/0x50 [ 4112.305681] do_syscall_64+0xc1/0x440 [ 4112.306967] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4112.309813] RIP: 0033:0x7f33957fc10b [ 4112.314816] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4112.325292] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4112.328966] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4112.334645] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4112.337224] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4112.339515] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4112.341635] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4116.061217] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4116.068607] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4116.109318] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4116.126173] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4116.134584] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4116.142468] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4116.150498] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4116.171819] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4116.179760] RAX: 0000000000004417 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4116.187400] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4116.191205] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4116.198113] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4116.206791] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4116.214131] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4116.224395] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4116.235655] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4116.242698] Call Trace: [ 4116.247195] [ 4116.248210] ? show_regs.cold.9+0x22/0x2f [ 4116.251160] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4116.256848] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4116.258327] ? watchdog+0x40/0x40 [ 4116.259316] ? hrtimer_interrupt+0x128/0x2c0 [ 4116.263606] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4116.273453] ? apic_timer_interrupt+0xf/0x20 [ 4116.276848] [ 4116.280718] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4116.286164] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4116.296159] do_raw_spin_lock+0x177/0x190 [ 4116.300293] _raw_spin_lock+0x21/0x30 [ 4116.303432] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4116.309450] process_one_work+0x2c8/0x700 [ 4116.310955] worker_thread+0x296/0x6e0 [ 4116.313868] ? rescuer_thread+0x570/0x570 [ 4116.318968] kthread+0x1d1/0x200 [ 4116.322334] ? set_kthread_struct+0x70/0x70 [ 4116.328531] ret_from_fork+0x1f/0x30 [ 4124.053444] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4124.057707] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4124.101783] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4124.112882] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4124.122118] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4124.124579] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4124.133882] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4124.137602] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4124.141145] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4124.145422] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4124.149242] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4124.152436] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4124.156053] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4124.159765] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4124.162344] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4124.166312] Call Trace: [ 4124.167438] [ 4124.169113] ? show_regs.cold.9+0x22/0x2f [ 4124.170792] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4124.173210] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4124.175436] ? watchdog+0x40/0x40 [ 4124.176900] ? hrtimer_interrupt+0x128/0x2c0 [ 4124.179059] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4124.181296] ? apic_timer_interrupt+0xf/0x20 [ 4124.184126] [ 4124.185177] ? native_safe_halt+0x2e/0x50 [ 4124.186569] kvm_wait+0xcd/0xe0 [ 4124.187631] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4124.189784] do_raw_spin_lock+0x177/0x190 [ 4124.191448] _raw_spin_lock+0x21/0x30 [ 4124.194250] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4124.197466] obd_mod_free+0x46/0x140 [ptlrpc] [ 4124.200821] mdc_commit_open+0x91/0xb0 [mdc] [ 4124.203241] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4124.206580] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4124.209595] after_reply+0xe2c/0x1780 [ptlrpc] [ 4124.212526] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4124.216163] ? __next_timer_interrupt+0x160/0x160 [ 4124.218774] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4124.221360] ? do_wait_intr+0xf0/0xf0 [ 4124.223363] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4124.227433] kthread+0x1d1/0x200 [ 4124.229083] ? set_kthread_struct+0x70/0x70 [ 4124.231405] ret_from_fork+0x1f/0x30 [ 4138.068227] rcu: INFO: rcu_sched self-detected stall on CPU [ 4138.073907] rcu: 0-....: (1268117 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=383755 [ 4138.094589] (t=2040048 jiffies g=297885 q=11313) [ 4138.114069] NMI backtrace for cpu 0 [ 4138.120730] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4138.130349] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4138.142313] Call Trace: [ 4138.143818] [ 4138.145883] ? dump_stack+0xbb/0x10e [ 4138.147962] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4138.157483] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4138.163612] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4138.167065] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4138.172147] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4138.175564] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4138.178663] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4138.180715] ? update_process_times+0x6d/0xc0 [ 4138.182336] ? tick_sched_handle.isra.18+0x27/0x90 [ 4138.185973] ? tick_sched_timer+0x53/0xb0 [ 4138.187991] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4138.189207] ? tick_sched_handle.isra.18+0x90/0x90 [ 4138.192882] ? hrtimer_interrupt+0x128/0x2c0 [ 4138.195375] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4138.197178] ? apic_timer_interrupt+0xf/0x20 [ 4138.199105] [ 4138.199927] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4138.202296] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4138.204115] ? do_raw_spin_lock+0x177/0x190 [ 4138.205920] ? _raw_spin_lock+0x21/0x30 [ 4138.207362] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4138.209377] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4138.211740] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4138.213462] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4138.219503] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4138.222114] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4138.224433] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4138.227307] ? __next_timer_interrupt+0x160/0x160 [ 4138.229384] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4138.231589] ? do_wait_intr+0xf0/0xf0 [ 4138.232863] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4138.235326] ? kthread+0x1d1/0x200 [ 4138.236620] ? set_kthread_struct+0x70/0x70 [ 4138.238407] ? ret_from_fork+0x1f/0x30 [ 4140.059061] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4140.062661] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4140.082954] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4140.089842] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4140.099261] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4140.105476] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4140.118293] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 4140.123894] RAX: 0000000000004233 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4140.128260] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4140.132772] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4140.135291] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4140.137984] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4140.140777] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4140.144957] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4140.147194] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4140.149347] Call Trace: [ 4140.149972] [ 4140.154700] ? show_regs.cold.9+0x22/0x2f [ 4140.156882] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4140.159690] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4140.163985] ? watchdog+0x40/0x40 [ 4140.166476] ? hrtimer_interrupt+0x128/0x2c0 [ 4140.168328] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4140.170272] ? apic_timer_interrupt+0xf/0x20 [ 4140.171957] [ 4140.172742] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4140.175085] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4140.176994] do_raw_spin_lock+0x177/0x190 [ 4140.181273] _raw_spin_lock+0x21/0x30 [ 4140.182649] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4140.189505] ? do_raw_spin_unlock+0x75/0x190 [ 4140.197724] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4140.202182] ? do_wait_intr+0xf0/0xf0 [ 4140.206729] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4140.209608] mdc_reint+0x2d/0x110 [mdc] [ 4140.212422] mdc_unlink+0x658/0x870 [mdc] [ 4140.216989] lmv_unlink+0x34d/0xbd0 [lmv] [ 4140.222417] ll_unlink+0x24d/0x7f0 [lustre] [ 4140.225126] vfs_unlink+0x202/0x330 [ 4140.226324] do_unlinkat+0x2fa/0x460 [ 4140.230633] __x64_sys_unlink+0x37/0x50 [ 4140.232346] do_syscall_64+0xc1/0x440 [ 4140.234275] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4140.239191] RIP: 0033:0x7f33957fc10b [ 4140.244160] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4140.256471] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4140.263232] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4140.268376] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4140.275360] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4140.286305] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4140.292753] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4144.058035] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4144.066386] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4144.086741] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4144.096893] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4144.101288] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4144.104927] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4144.106900] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4144.116630] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4144.120166] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4144.123692] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4144.126736] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4144.132163] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4144.135910] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4144.140563] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4144.145363] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4144.150018] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4144.153111] Call Trace: [ 4144.153969] [ 4144.154914] ? show_regs.cold.9+0x22/0x2f [ 4144.158286] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4144.160212] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4144.162683] ? watchdog+0x40/0x40 [ 4144.164252] ? hrtimer_interrupt+0x128/0x2c0 [ 4144.166151] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4144.168183] ? apic_timer_interrupt+0xf/0x20 [ 4144.170097] [ 4144.170959] ? native_safe_halt+0x2e/0x50 [ 4144.172849] kvm_wait+0xcd/0xe0 [ 4144.174235] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4144.176432] do_raw_spin_lock+0x177/0x190 [ 4144.177948] _raw_spin_lock+0x21/0x30 [ 4144.179635] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4144.182205] process_one_work+0x2c8/0x700 [ 4144.183610] worker_thread+0x296/0x6e0 [ 4144.185134] ? rescuer_thread+0x570/0x570 [ 4144.186818] kthread+0x1d1/0x200 [ 4144.188386] ? set_kthread_struct+0x70/0x70 [ 4144.190142] ret_from_fork+0x1f/0x30 [ 4164.053171] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 4164.058704] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4164.078581] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4164.082622] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4164.086163] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4164.088460] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4164.095632] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4164.098453] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4164.101386] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4164.104883] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4164.110873] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4164.118316] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4164.122153] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4164.128675] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4164.130702] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4164.134198] Call Trace: [ 4164.135275] [ 4164.136358] ? show_regs.cold.9+0x22/0x2f [ 4164.138038] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4164.141380] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4164.143262] ? watchdog+0x40/0x40 [ 4164.145001] ? hrtimer_interrupt+0x128/0x2c0 [ 4164.147671] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4164.151176] ? apic_timer_interrupt+0xf/0x20 [ 4164.152749] [ 4164.153800] ? native_safe_halt+0x2e/0x50 [ 4164.155493] kvm_wait+0xcd/0xe0 [ 4164.156726] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4164.160094] do_raw_spin_lock+0x177/0x190 [ 4164.163517] _raw_spin_lock+0x21/0x30 [ 4164.165705] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4164.168240] obd_mod_free+0x46/0x140 [ptlrpc] [ 4164.170534] mdc_commit_open+0x91/0xb0 [mdc] [ 4164.172441] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4164.174923] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4164.177798] after_reply+0xe2c/0x1780 [ptlrpc] [ 4164.179974] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4164.182621] ? __next_timer_interrupt+0x160/0x160 [ 4164.186359] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4164.188588] ? do_wait_intr+0xf0/0xf0 [ 4164.190689] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4164.193894] kthread+0x1d1/0x200 [ 4164.195861] ? set_kthread_struct+0x70/0x70 [ 4164.197618] ret_from_fork+0x1f/0x30 [ 4168.060086] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4168.067336] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4168.120170] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4168.130373] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4168.135376] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4168.139089] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4168.162086] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4168.174320] RAX: 0000000000004688 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4168.182154] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4168.187236] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4168.193334] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4168.197735] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4168.201672] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4168.205871] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4168.208427] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4168.212214] Call Trace: [ 4168.213194] [ 4168.214185] ? show_regs.cold.9+0x22/0x2f [ 4168.215692] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4168.217717] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4168.219255] ? watchdog+0x40/0x40 [ 4168.220280] ? hrtimer_interrupt+0x128/0x2c0 [ 4168.222171] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4168.223762] ? apic_timer_interrupt+0xf/0x20 [ 4168.225849] [ 4168.226781] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4168.228762] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4168.230889] do_raw_spin_lock+0x177/0x190 [ 4168.232421] _raw_spin_lock+0x21/0x30 [ 4168.233925] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4168.236212] ? do_raw_spin_unlock+0x75/0x190 [ 4168.238031] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4168.240348] ? do_wait_intr+0xf0/0xf0 [ 4168.241831] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4168.244305] mdc_reint+0x2d/0x110 [mdc] [ 4168.246159] mdc_unlink+0x658/0x870 [mdc] [ 4168.247925] lmv_unlink+0x34d/0xbd0 [lmv] [ 4168.249749] ll_unlink+0x24d/0x7f0 [lustre] [ 4168.251537] vfs_unlink+0x202/0x330 [ 4168.252803] do_unlinkat+0x2fa/0x460 [ 4168.254659] __x64_sys_unlink+0x37/0x50 [ 4168.256279] do_syscall_64+0xc1/0x440 [ 4168.257824] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4168.259828] RIP: 0033:0x7f33957fc10b [ 4168.261200] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4168.268135] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4168.270747] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4168.273693] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4168.277493] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4168.281283] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4168.287693] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4172.058176] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4172.062918] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4172.099267] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4172.103600] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4172.107577] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4172.111755] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4172.114798] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4172.124622] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4172.127807] RAX: 000000000000330a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4172.130868] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4172.136794] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4172.141426] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4172.145709] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4172.153145] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4172.161410] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4172.168402] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4172.173534] Call Trace: [ 4172.175571] [ 4172.177092] ? show_regs.cold.9+0x22/0x2f [ 4172.179233] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4172.183551] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4172.186405] ? watchdog+0x40/0x40 [ 4172.190448] ? hrtimer_interrupt+0x128/0x2c0 [ 4172.193551] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4172.197413] ? apic_timer_interrupt+0xf/0x20 [ 4172.199782] [ 4172.201212] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4172.205440] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4172.208433] do_raw_spin_lock+0x177/0x190 [ 4172.210529] _raw_spin_lock+0x21/0x30 [ 4172.213263] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4172.216658] process_one_work+0x2c8/0x700 [ 4172.223599] worker_thread+0x296/0x6e0 [ 4172.225822] ? rescuer_thread+0x570/0x570 [ 4172.228461] kthread+0x1d1/0x200 [ 4172.230379] ? set_kthread_struct+0x70/0x70 [ 4172.233116] ret_from_fork+0x1f/0x30 [ 4192.053051] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 4192.056137] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4192.085543] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4192.090746] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4192.094768] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4192.096866] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4192.106564] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 4192.114200] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4192.119756] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4192.123545] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4192.128277] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4192.135011] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4192.140262] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4192.144461] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4192.147565] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4192.153024] Call Trace: [ 4192.154092] [ 4192.155192] ? show_regs.cold.9+0x22/0x2f [ 4192.156842] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4192.159241] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4192.163338] ? watchdog+0x40/0x40 [ 4192.165675] ? hrtimer_interrupt+0x128/0x2c0 [ 4192.168533] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4192.171393] ? apic_timer_interrupt+0xf/0x20 [ 4192.173847] [ 4192.175346] ? native_safe_halt+0x2e/0x50 [ 4192.177401] kvm_wait+0xcd/0xe0 [ 4192.179759] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4192.182901] do_raw_spin_lock+0x177/0x190 [ 4192.185265] _raw_spin_lock+0x21/0x30 [ 4192.188774] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4192.192340] obd_mod_free+0x46/0x140 [ptlrpc] [ 4192.196568] mdc_commit_open+0x91/0xb0 [mdc] [ 4192.199410] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4192.202809] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4192.206400] after_reply+0xe2c/0x1780 [ptlrpc] [ 4192.209371] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4192.212759] ? __next_timer_interrupt+0x160/0x160 [ 4192.216347] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4192.218709] ? do_wait_intr+0xf0/0xf0 [ 4192.220595] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4192.225006] kthread+0x1d1/0x200 [ 4192.226788] ? set_kthread_struct+0x70/0x70 [ 4192.230741] ret_from_fork+0x1f/0x30 [ 4196.059111] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4196.061923] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4196.162610] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4196.176339] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4196.188367] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4196.195357] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4196.212197] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4196.225861] RAX: 0000000000003d18 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4196.235342] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4196.246682] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4196.256126] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4196.260930] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4196.266636] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4196.276647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4196.280596] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4196.289421] Call Trace: [ 4196.291189] [ 4196.293186] ? show_regs.cold.9+0x22/0x2f [ 4196.298529] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4196.308507] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4196.317610] ? watchdog+0x40/0x40 [ 4196.321513] ? hrtimer_interrupt+0x128/0x2c0 [ 4196.323004] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4196.330674] ? apic_timer_interrupt+0xf/0x20 [ 4196.335499] [ 4196.336451] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4196.340449] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4196.349048] do_raw_spin_lock+0x177/0x190 [ 4196.352307] _raw_spin_lock+0x21/0x30 [ 4196.355224] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4196.359297] ? do_raw_spin_unlock+0x75/0x190 [ 4196.364159] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4196.367165] ? do_wait_intr+0xf0/0xf0 [ 4196.370295] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4196.373117] mdc_reint+0x2d/0x110 [mdc] [ 4196.377614] mdc_unlink+0x658/0x870 [mdc] [ 4196.380202] lmv_unlink+0x34d/0xbd0 [lmv] [ 4196.387325] ll_unlink+0x24d/0x7f0 [lustre] [ 4196.394114] vfs_unlink+0x202/0x330 [ 4196.397192] do_unlinkat+0x2fa/0x460 [ 4196.400628] __x64_sys_unlink+0x37/0x50 [ 4196.402030] do_syscall_64+0xc1/0x440 [ 4196.414144] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4196.419094] RIP: 0033:0x7f33957fc10b [ 4196.423562] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4196.439887] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4196.453196] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4196.467515] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4196.471257] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4196.489458] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4196.498947] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4200.058030] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4200.062847] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4200.122973] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4200.134103] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4200.149747] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4200.157734] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4200.159808] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4200.179287] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4200.184100] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4200.194090] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4200.201563] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4200.207694] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4200.211925] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4200.215896] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4200.219589] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4200.221625] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4200.224101] Call Trace: [ 4200.224918] [ 4200.225603] ? show_regs.cold.9+0x22/0x2f [ 4200.226977] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4200.228822] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4200.230097] ? watchdog+0x40/0x40 [ 4200.230951] ? hrtimer_interrupt+0x128/0x2c0 [ 4200.232001] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4200.233834] ? apic_timer_interrupt+0xf/0x20 [ 4200.235481] [ 4200.236257] ? native_safe_halt+0x2e/0x50 [ 4200.237845] kvm_wait+0xcd/0xe0 [ 4200.240090] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4200.244135] do_raw_spin_lock+0x177/0x190 [ 4200.246118] _raw_spin_lock+0x21/0x30 [ 4200.249272] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4200.263145] process_one_work+0x2c8/0x700 [ 4200.265846] worker_thread+0x296/0x6e0 [ 4200.266834] ? rescuer_thread+0x570/0x570 [ 4200.276545] kthread+0x1d1/0x200 [ 4200.278626] ? set_kthread_struct+0x70/0x70 [ 4200.284030] ret_from_fork+0x1f/0x30 [ 4220.053177] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 4220.065640] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4220.090745] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4220.096689] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4220.103508] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4220.106294] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4220.123420] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4220.130953] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4220.137399] RDX: 000000000000706c RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4220.141563] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4220.144590] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4220.153407] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4220.164041] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4220.179734] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4220.201850] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4220.222608] Call Trace: [ 4220.224210] [ 4220.226893] ? show_regs.cold.9+0x22/0x2f [ 4220.242072] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4220.243719] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4220.251058] ? watchdog+0x40/0x40 [ 4220.254767] ? hrtimer_interrupt+0x128/0x2c0 [ 4220.259050] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4220.269847] ? apic_timer_interrupt+0xf/0x20 [ 4220.278777] [ 4220.282478] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4220.288935] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4220.302392] do_raw_spin_lock+0x177/0x190 [ 4220.305886] _raw_spin_lock+0x21/0x30 [ 4220.308722] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4220.317122] obd_mod_free+0x46/0x140 [ptlrpc] [ 4220.329225] mdc_commit_open+0x91/0xb0 [mdc] [ 4220.341122] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4220.343035] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4220.352573] after_reply+0xe2c/0x1780 [ptlrpc] [ 4220.359766] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4220.363787] ? __next_timer_interrupt+0x160/0x160 [ 4220.365637] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4220.368830] ? do_wait_intr+0xf0/0xf0 [ 4220.373045] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4220.376248] kthread+0x1d1/0x200 [ 4220.379730] ? set_kthread_struct+0x70/0x70 [ 4220.381745] ret_from_fork+0x1f/0x30 [ 4224.059023] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4224.062636] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4224.103119] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4224.107645] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4224.110951] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4224.112655] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4224.120464] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4224.126772] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4224.135165] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4224.141026] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4224.146611] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4224.151306] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4224.155199] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4224.163670] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4224.170904] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4224.174699] Call Trace: [ 4224.176730] [ 4224.177752] ? show_regs.cold.9+0x22/0x2f [ 4224.179744] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4224.183518] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4224.187104] ? watchdog+0x40/0x40 [ 4224.189689] ? hrtimer_interrupt+0x128/0x2c0 [ 4224.192068] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4224.198761] ? apic_timer_interrupt+0xf/0x20 [ 4224.201765] [ 4224.204683] ? native_safe_halt+0x2e/0x50 [ 4224.210680] kvm_wait+0xcd/0xe0 [ 4224.214469] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4224.221751] do_raw_spin_lock+0x177/0x190 [ 4224.225559] _raw_spin_lock+0x21/0x30 [ 4224.227385] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4224.237618] ? do_raw_spin_unlock+0x75/0x190 [ 4224.239047] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4224.247753] ? do_wait_intr+0xf0/0xf0 [ 4224.250390] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4224.258413] mdc_reint+0x2d/0x110 [mdc] [ 4224.262247] mdc_unlink+0x658/0x870 [mdc] [ 4224.267783] lmv_unlink+0x34d/0xbd0 [lmv] [ 4224.277494] ll_unlink+0x24d/0x7f0 [lustre] [ 4224.284065] vfs_unlink+0x202/0x330 [ 4224.287334] do_unlinkat+0x2fa/0x460 [ 4224.290236] __x64_sys_unlink+0x37/0x50 [ 4224.292652] do_syscall_64+0xc1/0x440 [ 4224.294675] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4224.297746] RIP: 0033:0x7f33957fc10b [ 4224.302192] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4224.317161] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4224.325183] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4224.330541] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4224.343655] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4224.351155] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4224.355748] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4228.060256] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4228.068062] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4228.129584] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4228.149300] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4228.158125] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4228.165626] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4228.170818] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4228.188627] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4228.198549] RAX: 0000000000007c1a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4228.203586] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4228.212361] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4228.218011] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4228.225645] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4228.238804] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4228.241303] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4228.255609] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4228.265113] Call Trace: [ 4228.266631] [ 4228.267530] ? show_regs.cold.9+0x22/0x2f [ 4228.272416] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4228.282870] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4228.290165] ? watchdog+0x40/0x40 [ 4228.291957] ? hrtimer_interrupt+0x128/0x2c0 [ 4228.294727] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4228.296213] ? apic_timer_interrupt+0xf/0x20 [ 4228.297751] [ 4228.298952] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4228.301496] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4228.304004] do_raw_spin_lock+0x177/0x190 [ 4228.307604] _raw_spin_lock+0x21/0x30 [ 4228.310473] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4228.314725] process_one_work+0x2c8/0x700 [ 4228.318387] worker_thread+0x296/0x6e0 [ 4228.320542] ? rescuer_thread+0x570/0x570 [ 4228.322976] kthread+0x1d1/0x200 [ 4228.324491] ? set_kthread_struct+0x70/0x70 [ 4228.327250] ret_from_fork+0x1f/0x30 [ 4248.053456] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 4248.070414] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4248.145218] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4248.165184] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4248.183251] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4248.188586] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4248.214972] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4248.224486] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4248.228949] RDX: 0000000000001447 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4248.234156] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4248.237355] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4248.243282] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4248.251991] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4248.258267] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4248.266323] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4248.278061] Call Trace: [ 4248.285534] [ 4248.286888] ? show_regs.cold.9+0x22/0x2f [ 4248.292456] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4248.297024] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4248.299148] ? watchdog+0x40/0x40 [ 4248.300231] ? hrtimer_interrupt+0x128/0x2c0 [ 4248.302892] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4248.305053] ? apic_timer_interrupt+0xf/0x20 [ 4248.311539] [ 4248.313155] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4248.318126] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4248.324787] do_raw_spin_lock+0x177/0x190 [ 4248.327874] _raw_spin_lock+0x21/0x30 [ 4248.333701] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4248.339197] obd_mod_free+0x46/0x140 [ptlrpc] [ 4248.346378] mdc_commit_open+0x91/0xb0 [mdc] [ 4248.352236] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4248.358938] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4248.365834] after_reply+0xe2c/0x1780 [ptlrpc] [ 4248.369855] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4248.372308] ? __next_timer_interrupt+0x160/0x160 [ 4248.378595] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4248.380530] ? do_wait_intr+0xf0/0xf0 [ 4248.382516] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4248.387397] kthread+0x1d1/0x200 [ 4248.388927] ? set_kthread_struct+0x70/0x70 [ 4248.392873] ret_from_fork+0x1f/0x30 [ 4252.062293] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4252.066689] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4252.116673] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4252.126277] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4252.132430] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4252.139736] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4252.155994] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4252.165749] RAX: 000000000000762d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4252.173539] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4252.179444] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4252.187607] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4252.195381] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4252.199397] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4252.206658] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4252.216583] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4252.230050] Call Trace: [ 4252.238531] [ 4252.244668] ? show_regs.cold.9+0x22/0x2f [ 4252.253782] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4252.258211] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4252.268771] ? watchdog+0x40/0x40 [ 4252.269910] ? hrtimer_interrupt+0x128/0x2c0 [ 4252.277489] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4252.279104] ? apic_timer_interrupt+0xf/0x20 [ 4252.283427] [ 4252.284751] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4252.287469] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4252.293713] do_raw_spin_lock+0x177/0x190 [ 4252.296469] _raw_spin_lock+0x21/0x30 [ 4252.298814] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4252.304907] ? do_raw_spin_unlock+0x75/0x190 [ 4252.308532] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4252.310344] ? do_wait_intr+0xf0/0xf0 [ 4252.313568] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4252.316775] mdc_reint+0x2d/0x110 [mdc] [ 4252.320870] mdc_unlink+0x658/0x870 [mdc] [ 4252.323788] lmv_unlink+0x34d/0xbd0 [lmv] [ 4252.326005] ll_unlink+0x24d/0x7f0 [lustre] [ 4252.328983] vfs_unlink+0x202/0x330 [ 4252.332024] do_unlinkat+0x2fa/0x460 [ 4252.334025] __x64_sys_unlink+0x37/0x50 [ 4252.336954] do_syscall_64+0xc1/0x440 [ 4252.338217] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4252.344046] RIP: 0033:0x7f33957fc10b [ 4252.346531] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4252.360894] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4252.368923] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4252.374702] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4252.387026] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4252.395466] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4252.402027] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4256.058089] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4256.062157] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4256.093099] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4256.097675] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4256.101400] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4256.104611] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4256.107115] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4256.114767] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4256.117566] RAX: 000000000000070b RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4256.120350] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4256.123554] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4256.126977] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4256.130047] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4256.133282] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4256.136411] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4256.139068] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4256.142114] Call Trace: [ 4256.143054] [ 4256.143732] ? show_regs.cold.9+0x22/0x2f [ 4256.145615] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4256.147654] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4256.149460] ? watchdog+0x40/0x40 [ 4256.150521] ? hrtimer_interrupt+0x128/0x2c0 [ 4256.152375] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4256.154651] ? apic_timer_interrupt+0xf/0x20 [ 4256.156807] [ 4256.157582] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4256.159933] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4256.162555] do_raw_spin_lock+0x177/0x190 [ 4256.164019] _raw_spin_lock+0x21/0x30 [ 4256.165975] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4256.168101] process_one_work+0x2c8/0x700 [ 4256.169739] worker_thread+0x296/0x6e0 [ 4256.172126] ? rescuer_thread+0x570/0x570 [ 4256.173726] kthread+0x1d1/0x200 [ 4256.175701] ? set_kthread_struct+0x70/0x70 [ 4256.177494] ret_from_fork+0x1f/0x30 [ 4276.053225] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4276.060564] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4276.087007] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4276.094700] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4276.101105] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4276.106589] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4276.119365] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4276.127758] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4276.134892] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4276.141970] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4276.147710] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4276.152277] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4276.158501] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4276.164179] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4276.171920] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4276.176677] Call Trace: [ 4276.178768] [ 4276.180664] ? show_regs.cold.9+0x22/0x2f [ 4276.185741] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4276.189470] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4276.193575] ? watchdog+0x40/0x40 [ 4276.195179] ? hrtimer_interrupt+0x128/0x2c0 [ 4276.198336] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4276.199745] ? apic_timer_interrupt+0xf/0x20 [ 4276.205001] [ 4276.205740] ? native_safe_halt+0x2e/0x50 [ 4276.208929] kvm_wait+0xcd/0xe0 [ 4276.210230] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4276.217766] do_raw_spin_lock+0x177/0x190 [ 4276.219097] _raw_spin_lock+0x21/0x30 [ 4276.222086] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4276.225987] obd_mod_free+0x46/0x140 [ptlrpc] [ 4276.229773] mdc_commit_open+0x91/0xb0 [mdc] [ 4276.235625] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4276.241079] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4276.246444] after_reply+0xe2c/0x1780 [ptlrpc] [ 4276.248407] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4276.254747] ? __next_timer_interrupt+0x160/0x160 [ 4276.259084] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4276.263604] ? do_wait_intr+0xf0/0xf0 [ 4276.266753] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4276.272137] kthread+0x1d1/0x200 [ 4276.274786] ? set_kthread_struct+0x70/0x70 [ 4276.278932] ret_from_fork+0x1f/0x30 [ 4280.059093] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4280.062169] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4280.107367] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4280.113609] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4280.120061] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4280.123482] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4280.134284] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4280.141148] RAX: 0000000000005136 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4280.145143] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4280.150825] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4280.155437] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4280.165992] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4280.168976] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4280.173988] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4280.179336] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4280.186275] Call Trace: [ 4280.187687] [ 4280.188513] ? show_regs.cold.9+0x22/0x2f [ 4280.191180] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4280.193965] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4280.202006] ? watchdog+0x40/0x40 [ 4280.203386] ? hrtimer_interrupt+0x128/0x2c0 [ 4280.207377] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4280.215458] ? apic_timer_interrupt+0xf/0x20 [ 4280.219083] [ 4280.221524] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4280.225300] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4280.231679] do_raw_spin_lock+0x177/0x190 [ 4280.233189] _raw_spin_lock+0x21/0x30 [ 4280.238380] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4280.245744] ? do_raw_spin_unlock+0x75/0x190 [ 4280.247414] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4280.252367] ? do_wait_intr+0xf0/0xf0 [ 4280.254403] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4280.257276] mdc_reint+0x2d/0x110 [mdc] [ 4280.259375] mdc_unlink+0x658/0x870 [mdc] [ 4280.263032] lmv_unlink+0x34d/0xbd0 [lmv] [ 4280.265173] ll_unlink+0x24d/0x7f0 [lustre] [ 4280.267224] vfs_unlink+0x202/0x330 [ 4280.269716] do_unlinkat+0x2fa/0x460 [ 4280.270933] __x64_sys_unlink+0x37/0x50 [ 4280.274196] do_syscall_64+0xc1/0x440 [ 4280.275636] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4280.279722] RIP: 0033:0x7f33957fc10b [ 4280.281938] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4280.293851] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4280.297239] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4280.300974] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4280.304309] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4280.308202] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4280.312074] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4284.060534] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4284.070393] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4284.106134] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4284.122853] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4284.134391] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4284.138004] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4284.141969] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4284.152875] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4284.161455] RAX: 0000000000007634 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4284.165435] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4284.169036] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4284.175802] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4284.180421] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4284.187473] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4284.194593] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4284.197930] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4284.202296] Call Trace: [ 4284.203377] [ 4284.204320] ? show_regs.cold.9+0x22/0x2f [ 4284.209465] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4284.211329] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4284.213978] ? watchdog+0x40/0x40 [ 4284.215282] ? hrtimer_interrupt+0x128/0x2c0 [ 4284.218446] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4284.221005] ? apic_timer_interrupt+0xf/0x20 [ 4284.223762] [ 4284.224860] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4284.227371] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4284.234010] do_raw_spin_lock+0x177/0x190 [ 4284.238783] _raw_spin_lock+0x21/0x30 [ 4284.240444] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4284.243425] process_one_work+0x2c8/0x700 [ 4284.245444] worker_thread+0x296/0x6e0 [ 4284.247327] ? rescuer_thread+0x570/0x570 [ 4284.253245] kthread+0x1d1/0x200 [ 4284.261190] ? set_kthread_struct+0x70/0x70 [ 4284.262672] ret_from_fork+0x1f/0x30 [ 4304.053016] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4304.056851] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4304.084930] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4304.090364] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4304.094395] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4304.096230] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4304.104546] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4304.110876] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4304.115695] RDX: 000000000000158f RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4304.118398] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4304.121554] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4304.124530] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4304.127794] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4304.131381] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4304.133905] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4304.136539] Call Trace: [ 4304.137736] [ 4304.138568] ? show_regs.cold.9+0x22/0x2f [ 4304.140325] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4304.141839] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4304.143758] ? watchdog+0x40/0x40 [ 4304.145124] ? hrtimer_interrupt+0x128/0x2c0 [ 4304.146942] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4304.148454] ? apic_timer_interrupt+0xf/0x20 [ 4304.150271] [ 4304.151041] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4304.152759] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4304.155386] do_raw_spin_lock+0x177/0x190 [ 4304.157510] _raw_spin_lock+0x21/0x30 [ 4304.158887] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4304.160930] obd_mod_free+0x46/0x140 [ptlrpc] [ 4304.162496] mdc_commit_open+0x91/0xb0 [mdc] [ 4304.164226] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4304.167201] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4304.169619] after_reply+0xe2c/0x1780 [ptlrpc] [ 4304.171811] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4304.173989] ? __next_timer_interrupt+0x160/0x160 [ 4304.175748] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4304.178431] ? do_wait_intr+0xf0/0xf0 [ 4304.180023] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4304.182641] kthread+0x1d1/0x200 [ 4304.183775] ? set_kthread_struct+0x70/0x70 [ 4304.185645] ret_from_fork+0x1f/0x30 [ 4308.061571] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4308.075654] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4308.125968] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4308.141708] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4308.149840] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4308.158353] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4308.176960] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4308.193989] RAX: 0000000000006181 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4308.203224] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4308.214542] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4308.220135] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4308.226033] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4308.237927] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4308.251005] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4308.257410] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4308.266916] Call Trace: [ 4308.268780] [ 4308.269757] ? show_regs.cold.9+0x22/0x2f [ 4308.272639] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4308.280785] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4308.282570] ? watchdog+0x40/0x40 [ 4308.288424] ? hrtimer_interrupt+0x128/0x2c0 [ 4308.294302] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4308.302982] ? apic_timer_interrupt+0xf/0x20 [ 4308.307799] [ 4308.313724] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4308.322436] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4308.330899] do_raw_spin_lock+0x177/0x190 [ 4308.335084] _raw_spin_lock+0x21/0x30 [ 4308.340139] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4308.342190] ? do_raw_spin_unlock+0x75/0x190 [ 4308.354950] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4308.358665] ? do_wait_intr+0xf0/0xf0 [ 4308.361613] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4308.373704] mdc_reint+0x2d/0x110 [mdc] [ 4308.382817] mdc_unlink+0x658/0x870 [mdc] [ 4308.391327] lmv_unlink+0x34d/0xbd0 [lmv] [ 4308.396441] ll_unlink+0x24d/0x7f0 [lustre] [ 4308.399669] vfs_unlink+0x202/0x330 [ 4308.404367] do_unlinkat+0x2fa/0x460 [ 4308.408371] __x64_sys_unlink+0x37/0x50 [ 4308.413755] do_syscall_64+0xc1/0x440 [ 4308.415206] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4308.424841] RIP: 0033:0x7f33957fc10b [ 4308.430150] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4308.465272] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4308.473359] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4308.484072] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4308.495736] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4308.503669] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4308.515889] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4312.061159] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4312.065024] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4312.097912] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4312.102525] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4312.109778] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4312.118764] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4312.124014] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4312.138537] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4312.142647] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4312.151425] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4312.157640] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4312.167492] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4312.180657] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4312.191405] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4312.201833] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4312.207287] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4312.213829] Call Trace: [ 4312.216407] [ 4312.217507] ? show_regs.cold.9+0x22/0x2f [ 4312.218989] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4312.225603] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4312.231964] ? watchdog+0x40/0x40 [ 4312.234841] ? hrtimer_interrupt+0x128/0x2c0 [ 4312.238382] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4312.243977] ? apic_timer_interrupt+0xf/0x20 [ 4312.251492] [ 4312.255909] ? native_safe_halt+0x2e/0x50 [ 4312.260584] kvm_wait+0xcd/0xe0 [ 4312.267401] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4312.276279] do_raw_spin_lock+0x177/0x190 [ 4312.281718] _raw_spin_lock+0x21/0x30 [ 4312.284035] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4312.290697] process_one_work+0x2c8/0x700 [ 4312.296530] worker_thread+0x296/0x6e0 [ 4312.299950] ? rescuer_thread+0x570/0x570 [ 4312.308993] kthread+0x1d1/0x200 [ 4312.311745] ? set_kthread_struct+0x70/0x70 [ 4312.313187] ret_from_fork+0x1f/0x30 [ 4318.072635] rcu: INFO: rcu_sched self-detected stall on CPU [ 4318.084650] rcu: 0-....: (1383149 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=418082 [ 4318.095072] (t=2220052 jiffies g=297885 q=12008) [ 4318.098775] NMI backtrace for cpu 0 [ 4318.104812] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4318.115234] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4318.122458] Call Trace: [ 4318.126594] [ 4318.132365] ? dump_stack+0xbb/0x10e [ 4318.133632] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4318.140632] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4318.147759] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4318.155617] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4318.160639] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4318.164667] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4318.166913] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4318.170626] ? update_process_times+0x6d/0xc0 [ 4318.173835] ? tick_sched_handle.isra.18+0x27/0x90 [ 4318.179480] ? tick_sched_timer+0x53/0xb0 [ 4318.182497] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4318.184843] ? tick_sched_handle.isra.18+0x90/0x90 [ 4318.187784] ? hrtimer_interrupt+0x128/0x2c0 [ 4318.190788] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4318.195228] ? apic_timer_interrupt+0xf/0x20 [ 4318.205070] [ 4318.206021] ? native_safe_halt+0x2e/0x50 [ 4318.207414] ? kvm_wait+0xcd/0xe0 [ 4318.213688] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4318.220194] ? do_raw_spin_lock+0x177/0x190 [ 4318.225461] ? _raw_spin_lock+0x21/0x30 [ 4318.228554] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4318.232637] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4318.236499] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4318.239129] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4318.242828] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4318.246582] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4318.250349] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4318.254049] ? __next_timer_interrupt+0x160/0x160 [ 4318.256845] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4318.260492] ? do_wait_intr+0xf0/0xf0 [ 4318.262201] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4318.270570] ? kthread+0x1d1/0x200 [ 4318.272455] ? set_kthread_struct+0x70/0x70 [ 4318.279176] ? ret_from_fork+0x1f/0x30 [ 4336.061671] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4336.073858] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4336.143787] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4336.158570] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4336.166676] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4336.176169] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4336.195580] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4336.209553] RAX: 0000000000003609 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4336.219279] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4336.226280] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4336.231109] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4336.235376] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4336.239433] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4336.247190] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4336.249293] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4336.260108] Call Trace: [ 4336.264501] [ 4336.268000] ? show_regs.cold.9+0x22/0x2f [ 4336.274092] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4336.280286] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4336.286450] ? watchdog+0x40/0x40 [ 4336.293337] ? hrtimer_interrupt+0x128/0x2c0 [ 4336.294990] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4336.304490] ? apic_timer_interrupt+0xf/0x20 [ 4336.312465] [ 4336.313191] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4336.320199] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4336.323088] do_raw_spin_lock+0x177/0x190 [ 4336.327557] _raw_spin_lock+0x21/0x30 [ 4336.328803] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4336.341117] ? do_raw_spin_unlock+0x75/0x190 [ 4336.348304] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4336.351507] ? do_wait_intr+0xf0/0xf0 [ 4336.354223] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4336.360287] mdc_reint+0x2d/0x110 [mdc] [ 4336.363044] mdc_unlink+0x658/0x870 [mdc] [ 4336.365587] lmv_unlink+0x34d/0xbd0 [lmv] [ 4336.368558] ll_unlink+0x24d/0x7f0 [lustre] [ 4336.370804] vfs_unlink+0x202/0x330 [ 4336.374148] do_unlinkat+0x2fa/0x460 [ 4336.376337] __x64_sys_unlink+0x37/0x50 [ 4336.380453] do_syscall_64+0xc1/0x440 [ 4336.382714] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4336.389270] RIP: 0033:0x7f33957fc10b [ 4336.393836] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4336.424050] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4336.429858] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4336.437549] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4336.441488] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4336.446046] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4336.452131] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4340.059245] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4340.065730] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4340.101283] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4340.110350] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4340.117736] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4340.128999] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30a/0x6d0 [ 4340.134394] Code: 03 00 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 90 83 e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 [ 4340.153087] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4340.156086] RAX: 0000000000003348 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4340.158999] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4340.162945] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4340.166391] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4340.171489] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4340.176688] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4340.183496] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4340.187081] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4340.189902] Call Trace: [ 4340.191470] [ 4340.192250] ? show_regs.cold.9+0x22/0x2f [ 4340.195437] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4340.201460] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4340.204087] ? watchdog+0x40/0x40 [ 4340.205040] ? hrtimer_interrupt+0x128/0x2c0 [ 4340.206871] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4340.208180] ? apic_timer_interrupt+0xf/0x20 [ 4340.210094] [ 4340.210638] ? __pv_queued_spin_lock_slowpath+0x30a/0x6d0 [ 4340.214394] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4340.222007] do_raw_spin_lock+0x177/0x190 [ 4340.225384] _raw_spin_lock+0x21/0x30 [ 4340.226852] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4340.229701] process_one_work+0x2c8/0x700 [ 4340.232932] worker_thread+0x296/0x6e0 [ 4340.235030] ? rescuer_thread+0x570/0x570 [ 4340.239251] kthread+0x1d1/0x200 [ 4340.241311] ? set_kthread_struct+0x70/0x70 [ 4340.242641] ret_from_fork+0x1f/0x30 [ 4344.056617] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4344.063120] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4344.110971] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4344.124236] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4344.131103] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4344.133364] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4344.147557] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4344.154287] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4344.163515] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4344.169725] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4344.175906] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4344.182007] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4344.185168] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4344.194413] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4344.197383] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4344.203946] Call Trace: [ 4344.208175] [ 4344.209107] ? show_regs.cold.9+0x22/0x2f [ 4344.213319] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4344.216073] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4344.222230] ? watchdog+0x40/0x40 [ 4344.223518] ? hrtimer_interrupt+0x128/0x2c0 [ 4344.228763] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4344.231020] ? apic_timer_interrupt+0xf/0x20 [ 4344.232715] [ 4344.237288] ? native_safe_halt+0x2e/0x50 [ 4344.238697] kvm_wait+0xcd/0xe0 [ 4344.242403] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4344.245339] do_raw_spin_lock+0x177/0x190 [ 4344.250236] _raw_spin_lock+0x21/0x30 [ 4344.253612] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4344.257987] obd_mod_free+0x46/0x140 [ptlrpc] [ 4344.263161] mdc_commit_open+0x91/0xb0 [mdc] [ 4344.267935] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4344.270496] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4344.276393] after_reply+0xe2c/0x1780 [ptlrpc] [ 4344.281673] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4344.286801] ? __next_timer_interrupt+0x160/0x160 [ 4344.292298] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4344.294846] ? do_wait_intr+0xf0/0xf0 [ 4344.299364] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4344.305510] kthread+0x1d1/0x200 [ 4344.306753] ? set_kthread_struct+0x70/0x70 [ 4344.311721] ret_from_fork+0x1f/0x30 [ 4364.066339] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4364.077344] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4364.132425] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4364.148023] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4364.161031] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4364.167340] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4364.178400] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4364.182574] RAX: 0000000000007b81 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4364.186008] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4364.188149] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4364.191531] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4364.194558] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4364.198267] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4364.202181] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4364.204772] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4364.207973] Call Trace: [ 4364.209147] [ 4364.209954] ? show_regs.cold.9+0x22/0x2f [ 4364.211702] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4364.213865] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4364.216273] ? watchdog+0x40/0x40 [ 4364.221613] ? hrtimer_interrupt+0x128/0x2c0 [ 4364.226310] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4364.231210] ? apic_timer_interrupt+0xf/0x20 [ 4364.234960] [ 4364.236030] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4364.238223] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4364.240940] do_raw_spin_lock+0x177/0x190 [ 4364.242978] _raw_spin_lock+0x21/0x30 [ 4364.244871] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4364.248749] ? do_raw_spin_unlock+0x75/0x190 [ 4364.250775] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4364.253580] ? do_wait_intr+0xf0/0xf0 [ 4364.255298] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4364.257934] mdc_reint+0x2d/0x110 [mdc] [ 4364.259919] mdc_unlink+0x658/0x870 [mdc] [ 4364.261386] lmv_unlink+0x34d/0xbd0 [lmv] [ 4364.262640] ll_unlink+0x24d/0x7f0 [lustre] [ 4364.264272] vfs_unlink+0x202/0x330 [ 4364.266951] do_unlinkat+0x2fa/0x460 [ 4364.269164] __x64_sys_unlink+0x37/0x50 [ 4364.270392] do_syscall_64+0xc1/0x440 [ 4364.276212] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4364.280959] RIP: 0033:0x7f33957fc10b [ 4364.288133] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4364.305391] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4364.309021] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4364.315877] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4364.322291] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4364.335746] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4364.343536] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4368.060923] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4368.064418] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4368.086988] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4368.091200] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4368.098655] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4368.102524] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4368.107182] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4368.121450] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4368.126230] RAX: 0000000000006da1 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4368.132139] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4368.138240] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4368.142085] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4368.144946] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4368.150298] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4368.155461] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4368.159947] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4368.169944] Call Trace: [ 4368.170619] [ 4368.171288] ? show_regs.cold.9+0x22/0x2f [ 4368.176413] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4368.183275] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4368.185347] ? watchdog+0x40/0x40 [ 4368.187260] ? hrtimer_interrupt+0x128/0x2c0 [ 4368.188898] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4368.191469] ? apic_timer_interrupt+0xf/0x20 [ 4368.193564] [ 4368.195782] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4368.199165] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4368.200583] do_raw_spin_lock+0x177/0x190 [ 4368.201951] _raw_spin_lock+0x21/0x30 [ 4368.203406] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4368.205893] process_one_work+0x2c8/0x700 [ 4368.207915] worker_thread+0x296/0x6e0 [ 4368.209601] ? rescuer_thread+0x570/0x570 [ 4368.211646] kthread+0x1d1/0x200 [ 4368.213241] ? set_kthread_struct+0x70/0x70 [ 4368.215250] ret_from_fork+0x1f/0x30 [ 4372.053044] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4372.057300] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4372.094103] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4372.112061] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4372.127213] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4372.132378] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4372.166825] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4372.173350] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4372.187145] RDX: 00000000000002e4 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4372.200786] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4372.211248] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4372.216981] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4372.221831] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4372.229812] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4372.237862] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4372.245919] Call Trace: [ 4372.246887] [ 4372.247662] ? show_regs.cold.9+0x22/0x2f [ 4372.256367] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4372.265635] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4372.270662] ? watchdog+0x40/0x40 [ 4372.277265] ? hrtimer_interrupt+0x128/0x2c0 [ 4372.280340] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4372.287819] ? apic_timer_interrupt+0xf/0x20 [ 4372.292849] [ 4372.296481] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4372.298541] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4372.304003] do_raw_spin_lock+0x177/0x190 [ 4372.308108] _raw_spin_lock+0x21/0x30 [ 4372.310307] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4372.313783] obd_mod_free+0x46/0x140 [ptlrpc] [ 4372.323814] mdc_commit_open+0x91/0xb0 [mdc] [ 4372.328778] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4372.335417] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4372.340928] after_reply+0xe2c/0x1780 [ptlrpc] [ 4372.344726] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4372.348801] ? __next_timer_interrupt+0x160/0x160 [ 4372.351444] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4372.354682] ? do_wait_intr+0xf0/0xf0 [ 4372.357373] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4372.361201] kthread+0x1d1/0x200 [ 4372.362733] ? set_kthread_struct+0x70/0x70 [ 4372.365320] ret_from_fork+0x1f/0x30 [ 4392.059345] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4392.066942] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4392.107907] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4392.113947] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4392.120342] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4392.122738] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4392.136778] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4392.142355] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4392.148009] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4392.153673] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4392.158609] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4392.163906] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4392.174447] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4392.179463] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4392.186422] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4392.192936] Call Trace: [ 4392.194362] [ 4392.196117] ? show_regs.cold.9+0x22/0x2f [ 4392.197623] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4392.204045] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4392.206402] ? watchdog+0x40/0x40 [ 4392.208235] ? hrtimer_interrupt+0x128/0x2c0 [ 4392.210871] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4392.213729] ? apic_timer_interrupt+0xf/0x20 [ 4392.218337] [ 4392.220698] ? native_safe_halt+0x2e/0x50 [ 4392.223351] kvm_wait+0xcd/0xe0 [ 4392.227634] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4392.230725] do_raw_spin_lock+0x177/0x190 [ 4392.233069] _raw_spin_lock+0x21/0x30 [ 4392.236703] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4392.242815] ? do_raw_spin_unlock+0x75/0x190 [ 4392.246444] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4392.250689] ? do_wait_intr+0xf0/0xf0 [ 4392.253328] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4392.260826] mdc_reint+0x2d/0x110 [mdc] [ 4392.264064] mdc_unlink+0x658/0x870 [mdc] [ 4392.265760] lmv_unlink+0x34d/0xbd0 [lmv] [ 4392.268058] ll_unlink+0x24d/0x7f0 [lustre] [ 4392.270751] vfs_unlink+0x202/0x330 [ 4392.272825] do_unlinkat+0x2fa/0x460 [ 4392.274691] __x64_sys_unlink+0x37/0x50 [ 4392.276717] do_syscall_64+0xc1/0x440 [ 4392.278691] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4392.281317] RIP: 0033:0x7f33957fc10b [ 4392.283198] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4392.292740] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4392.300805] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4392.305697] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4392.309117] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4392.316674] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4392.318666] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4396.058498] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4396.070359] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4396.120553] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4396.128773] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4396.136298] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4396.142918] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4396.147941] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4396.157299] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4396.162399] RAX: 0000000000005f07 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4396.168222] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4396.177774] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4396.181446] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4396.200167] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4396.207261] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4396.214966] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4396.221039] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4396.225337] Call Trace: [ 4396.226594] [ 4396.228203] ? show_regs.cold.9+0x22/0x2f [ 4396.231602] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4396.238333] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4396.242523] ? watchdog+0x40/0x40 [ 4396.244318] ? hrtimer_interrupt+0x128/0x2c0 [ 4396.248131] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4396.250335] ? apic_timer_interrupt+0xf/0x20 [ 4396.253339] [ 4396.254783] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4396.264328] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4396.268996] do_raw_spin_lock+0x177/0x190 [ 4396.271370] _raw_spin_lock+0x21/0x30 [ 4396.274458] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4396.277629] process_one_work+0x2c8/0x700 [ 4396.282326] worker_thread+0x296/0x6e0 [ 4396.284899] ? rescuer_thread+0x570/0x570 [ 4396.287232] kthread+0x1d1/0x200 [ 4396.294990] ? set_kthread_struct+0x70/0x70 [ 4396.299915] ret_from_fork+0x1f/0x30 [ 4400.056689] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4400.061980] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4400.086092] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4400.090007] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4400.092729] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4400.094728] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4400.101004] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4400.103649] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4400.106149] RDX: 0000000000005d79 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4400.108729] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4400.111497] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4400.115783] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4400.124311] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4400.129710] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4400.134555] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4400.137379] Call Trace: [ 4400.142636] [ 4400.143326] ? show_regs.cold.9+0x22/0x2f [ 4400.145516] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4400.148470] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4400.150806] ? watchdog+0x40/0x40 [ 4400.152256] ? hrtimer_interrupt+0x128/0x2c0 [ 4400.154611] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4400.161464] ? apic_timer_interrupt+0xf/0x20 [ 4400.168026] [ 4400.168724] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4400.178156] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4400.181572] do_raw_spin_lock+0x177/0x190 [ 4400.186194] _raw_spin_lock+0x21/0x30 [ 4400.189138] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4400.196236] obd_mod_free+0x46/0x140 [ptlrpc] [ 4400.202176] mdc_commit_open+0x91/0xb0 [mdc] [ 4400.206107] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4400.208739] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4400.214682] after_reply+0xe2c/0x1780 [ptlrpc] [ 4400.218565] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4400.224300] ? __next_timer_interrupt+0x160/0x160 [ 4400.226237] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4400.229784] ? do_wait_intr+0xf0/0xf0 [ 4400.233013] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4400.237276] kthread+0x1d1/0x200 [ 4400.240081] ? set_kthread_struct+0x70/0x70 [ 4400.249698] ret_from_fork+0x1f/0x30 [ 4420.060497] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4420.070703] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4420.125271] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4420.136195] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4420.146518] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4420.161173] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4420.183894] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4420.195505] RAX: 0000000000006c04 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4420.203702] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4420.211175] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4420.226030] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4420.239526] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4420.249221] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4420.252687] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4420.254616] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4420.257596] Call Trace: [ 4420.258612] [ 4420.259296] ? show_regs.cold.9+0x22/0x2f [ 4420.260746] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4420.262307] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4420.264154] ? watchdog+0x40/0x40 [ 4420.265378] ? hrtimer_interrupt+0x128/0x2c0 [ 4420.267306] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4420.269226] ? apic_timer_interrupt+0xf/0x20 [ 4420.272555] [ 4420.273298] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4420.275768] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4420.277830] do_raw_spin_lock+0x177/0x190 [ 4420.279589] _raw_spin_lock+0x21/0x30 [ 4420.281183] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4420.283816] ? do_raw_spin_unlock+0x75/0x190 [ 4420.285784] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4420.288785] ? do_wait_intr+0xf0/0xf0 [ 4420.290377] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4420.293302] mdc_reint+0x2d/0x110 [mdc] [ 4420.294793] mdc_unlink+0x658/0x870 [mdc] [ 4420.296485] lmv_unlink+0x34d/0xbd0 [lmv] [ 4420.298203] ll_unlink+0x24d/0x7f0 [lustre] [ 4420.299854] vfs_unlink+0x202/0x330 [ 4420.303031] do_unlinkat+0x2fa/0x460 [ 4420.304733] __x64_sys_unlink+0x37/0x50 [ 4420.306880] do_syscall_64+0xc1/0x440 [ 4420.308584] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4420.310245] RIP: 0033:0x7f33957fc10b [ 4420.311607] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4420.320623] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4420.326227] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4420.329308] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4420.332113] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4420.336302] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4420.340214] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4424.059137] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4424.063396] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4424.118760] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4424.127774] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4424.139698] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4424.148103] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4424.160628] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4424.187203] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4424.198831] RAX: 0000000000007983 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4424.204551] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4424.207767] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4424.210798] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4424.220864] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4424.229212] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4424.234793] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4424.237842] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4424.241864] Call Trace: [ 4424.246173] [ 4424.247117] ? show_regs.cold.9+0x22/0x2f [ 4424.253217] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4424.257757] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4424.262094] ? watchdog+0x40/0x40 [ 4424.264733] ? hrtimer_interrupt+0x128/0x2c0 [ 4424.268745] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4424.273651] ? apic_timer_interrupt+0xf/0x20 [ 4424.275487] [ 4424.276298] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4424.281707] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4424.290377] do_raw_spin_lock+0x177/0x190 [ 4424.292551] _raw_spin_lock+0x21/0x30 [ 4424.294679] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4424.297342] process_one_work+0x2c8/0x700 [ 4424.304117] worker_thread+0x296/0x6e0 [ 4424.307700] ? rescuer_thread+0x570/0x570 [ 4424.311180] kthread+0x1d1/0x200 [ 4424.315814] ? set_kthread_struct+0x70/0x70 [ 4424.320215] ret_from_fork+0x1f/0x30 [ 4428.053133] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4428.057472] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4428.090321] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4428.097215] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4428.100557] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4428.103158] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4428.112977] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4428.115690] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4428.118354] RDX: 0000000000000321 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4428.121987] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4428.125419] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4428.129176] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4428.133014] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4428.137072] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4428.140158] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4428.142837] Call Trace: [ 4428.143731] [ 4428.144506] ? show_regs.cold.9+0x22/0x2f [ 4428.145999] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4428.147513] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4428.149184] ? watchdog+0x40/0x40 [ 4428.150499] ? hrtimer_interrupt+0x128/0x2c0 [ 4428.152519] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4428.157354] ? apic_timer_interrupt+0xf/0x20 [ 4428.159407] [ 4428.160519] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4428.164187] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4428.169473] do_raw_spin_lock+0x177/0x190 [ 4428.171720] _raw_spin_lock+0x21/0x30 [ 4428.173865] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4428.177864] obd_mod_free+0x46/0x140 [ptlrpc] [ 4428.187162] mdc_commit_open+0x91/0xb0 [mdc] [ 4428.190255] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4428.194699] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4428.202459] after_reply+0xe2c/0x1780 [ptlrpc] [ 4428.209562] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4428.213498] ? __next_timer_interrupt+0x160/0x160 [ 4428.218421] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4428.228569] ? do_wait_intr+0xf0/0xf0 [ 4428.231567] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4428.236477] kthread+0x1d1/0x200 [ 4428.237703] ? set_kthread_struct+0x70/0x70 [ 4428.239873] ret_from_fork+0x1f/0x30 [ 4448.060075] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4448.065895] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4448.093287] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4448.100367] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4448.104806] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4448.108932] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4448.120364] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4448.124094] RAX: 0000000000004e23 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4448.127484] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4448.130918] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4448.135726] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4448.139975] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4448.143894] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4448.148388] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4448.151021] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4448.155673] Call Trace: [ 4448.156678] [ 4448.157702] ? show_regs.cold.9+0x22/0x2f [ 4448.160152] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4448.162938] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4448.165153] ? watchdog+0x40/0x40 [ 4448.167194] ? hrtimer_interrupt+0x128/0x2c0 [ 4448.169250] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4448.172419] ? apic_timer_interrupt+0xf/0x20 [ 4448.175113] [ 4448.176072] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4448.180950] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4448.184225] do_raw_spin_lock+0x177/0x190 [ 4448.186835] _raw_spin_lock+0x21/0x30 [ 4448.189024] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4448.192197] ? do_raw_spin_unlock+0x75/0x190 [ 4448.194509] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4448.197954] ? do_wait_intr+0xf0/0xf0 [ 4448.199532] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4448.202537] mdc_reint+0x2d/0x110 [mdc] [ 4448.204824] mdc_unlink+0x658/0x870 [mdc] [ 4448.206853] lmv_unlink+0x34d/0xbd0 [lmv] [ 4448.208958] ll_unlink+0x24d/0x7f0 [lustre] [ 4448.211314] vfs_unlink+0x202/0x330 [ 4448.213074] do_unlinkat+0x2fa/0x460 [ 4448.215019] __x64_sys_unlink+0x37/0x50 [ 4448.216876] do_syscall_64+0xc1/0x440 [ 4448.218886] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4448.221323] RIP: 0033:0x7f33957fc10b [ 4448.223136] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4448.233169] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4448.237916] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4448.241293] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4448.245139] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4448.248664] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4448.251913] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4452.058080] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4452.061222] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4452.093962] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4452.098906] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4452.102979] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4452.107216] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4452.108613] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4452.121111] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4452.124321] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4452.126488] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4452.130822] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4452.137234] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4452.140057] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4452.141888] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4452.146962] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4452.149897] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4452.152703] Call Trace: [ 4452.156723] [ 4452.158084] ? show_regs.cold.9+0x22/0x2f [ 4452.160657] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4452.162879] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4452.164936] ? watchdog+0x40/0x40 [ 4452.166152] ? hrtimer_interrupt+0x128/0x2c0 [ 4452.167856] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4452.171701] ? apic_timer_interrupt+0xf/0x20 [ 4452.173604] [ 4452.174433] ? native_safe_halt+0x2e/0x50 [ 4452.178015] kvm_wait+0xcd/0xe0 [ 4452.179219] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4452.184945] do_raw_spin_lock+0x177/0x190 [ 4452.187888] _raw_spin_lock+0x21/0x30 [ 4452.189044] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4452.198905] process_one_work+0x2c8/0x700 [ 4452.202173] worker_thread+0x296/0x6e0 [ 4452.205538] ? rescuer_thread+0x570/0x570 [ 4452.207982] kthread+0x1d1/0x200 [ 4452.209067] ? set_kthread_struct+0x70/0x70 [ 4452.211735] ret_from_fork+0x1f/0x30 [ 4456.053280] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 4456.059470] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4456.095088] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4456.101767] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4456.105916] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4456.112215] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4456.125671] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4456.132939] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4456.137400] RDX: 0000000000000d50 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4456.140843] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4456.145324] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4456.151340] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4456.158342] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4456.169237] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4456.171010] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4456.173274] Call Trace: [ 4456.174175] [ 4456.176497] ? show_regs.cold.9+0x22/0x2f [ 4456.179661] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4456.184464] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4456.186397] ? watchdog+0x40/0x40 [ 4456.188343] ? hrtimer_interrupt+0x128/0x2c0 [ 4456.190157] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4456.192843] ? apic_timer_interrupt+0xf/0x20 [ 4456.194346] [ 4456.198832] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4456.203107] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4456.207248] do_raw_spin_lock+0x177/0x190 [ 4456.210516] _raw_spin_lock+0x21/0x30 [ 4456.212368] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4456.215434] obd_mod_free+0x46/0x140 [ptlrpc] [ 4456.218898] mdc_commit_open+0x91/0xb0 [mdc] [ 4456.220960] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4456.226538] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4456.234344] after_reply+0xe2c/0x1780 [ptlrpc] [ 4456.238569] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4456.244253] ? __next_timer_interrupt+0x160/0x160 [ 4456.246366] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4456.249158] ? do_wait_intr+0xf0/0xf0 [ 4456.251454] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4456.256091] kthread+0x1d1/0x200 [ 4456.257765] ? set_kthread_struct+0x70/0x70 [ 4456.259901] ret_from_fork+0x1f/0x30 [ 4476.059026] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4476.062399] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4476.131837] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4476.144516] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4476.150544] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4476.155814] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4476.164991] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4476.176548] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4476.181430] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4476.187485] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4476.191425] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4476.199027] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4476.205157] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4476.221831] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4476.226736] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4476.234043] Call Trace: [ 4476.236197] [ 4476.237212] ? show_regs.cold.9+0x22/0x2f [ 4476.240717] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4476.243623] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4476.245301] ? watchdog+0x40/0x40 [ 4476.246540] ? hrtimer_interrupt+0x128/0x2c0 [ 4476.249084] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4476.251508] ? apic_timer_interrupt+0xf/0x20 [ 4476.255069] [ 4476.255859] ? native_safe_halt+0x2e/0x50 [ 4476.258519] kvm_wait+0xcd/0xe0 [ 4476.260287] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4476.262166] do_raw_spin_lock+0x177/0x190 [ 4476.263530] _raw_spin_lock+0x21/0x30 [ 4476.264744] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4476.266737] ? do_raw_spin_unlock+0x75/0x190 [ 4476.268218] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4476.272142] ? do_wait_intr+0xf0/0xf0 [ 4476.275780] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4476.278395] mdc_reint+0x2d/0x110 [mdc] [ 4476.281319] mdc_unlink+0x658/0x870 [mdc] [ 4476.283498] lmv_unlink+0x34d/0xbd0 [lmv] [ 4476.286525] ll_unlink+0x24d/0x7f0 [lustre] [ 4476.289939] vfs_unlink+0x202/0x330 [ 4476.296903] do_unlinkat+0x2fa/0x460 [ 4476.298370] __x64_sys_unlink+0x37/0x50 [ 4476.301376] do_syscall_64+0xc1/0x440 [ 4476.308031] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4476.311175] RIP: 0033:0x7f33957fc10b [ 4476.314854] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4476.327701] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4476.340250] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4476.350446] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4476.363651] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4476.368237] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4476.375714] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4480.058074] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4480.061493] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4480.109255] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4480.118689] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4480.133671] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4480.141027] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4480.146027] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4480.163078] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4480.170008] RAX: 000000000000320c RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4480.177212] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4480.185604] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4480.196017] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4480.204969] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4480.210688] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4480.216640] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4480.220517] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4480.226599] Call Trace: [ 4480.230660] [ 4480.232717] ? show_regs.cold.9+0x22/0x2f [ 4480.236055] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4480.247934] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4480.255020] ? watchdog+0x40/0x40 [ 4480.257176] ? hrtimer_interrupt+0x128/0x2c0 [ 4480.259232] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4480.271810] ? apic_timer_interrupt+0xf/0x20 [ 4480.277331] [ 4480.278488] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4480.281109] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4480.292900] do_raw_spin_lock+0x177/0x190 [ 4480.297119] _raw_spin_lock+0x21/0x30 [ 4480.298488] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4480.304331] process_one_work+0x2c8/0x700 [ 4480.305789] worker_thread+0x296/0x6e0 [ 4480.320225] ? rescuer_thread+0x570/0x570 [ 4480.330884] kthread+0x1d1/0x200 [ 4480.332112] ? set_kthread_struct+0x70/0x70 [ 4480.344340] ret_from_fork+0x1f/0x30 [ 4484.053303] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 4484.064956] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4484.104290] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4484.112454] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4484.117575] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4484.120386] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4484.153851] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4484.162496] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4484.171482] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4484.180971] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4484.185024] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4484.190462] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4484.198343] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4484.201544] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4484.206594] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4484.210323] Call Trace: [ 4484.211757] [ 4484.212702] ? show_regs.cold.9+0x22/0x2f [ 4484.214598] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4484.216601] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4484.218228] ? watchdog+0x40/0x40 [ 4484.219806] ? hrtimer_interrupt+0x128/0x2c0 [ 4484.221611] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4484.227609] ? apic_timer_interrupt+0xf/0x20 [ 4484.229500] [ 4484.230051] ? native_safe_halt+0x2e/0x50 [ 4484.231171] kvm_wait+0xcd/0xe0 [ 4484.234911] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4484.236715] do_raw_spin_lock+0x177/0x190 [ 4484.239165] _raw_spin_lock+0x21/0x30 [ 4484.241245] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4484.244917] obd_mod_free+0x46/0x140 [ptlrpc] [ 4484.247982] mdc_commit_open+0x91/0xb0 [mdc] [ 4484.250575] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4484.258957] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4484.261773] after_reply+0xe2c/0x1780 [ptlrpc] [ 4484.264656] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4484.267498] ? __next_timer_interrupt+0x160/0x160 [ 4484.270018] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4484.272280] ? do_wait_intr+0xf0/0xf0 [ 4484.273992] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4484.277222] kthread+0x1d1/0x200 [ 4484.278923] ? set_kthread_struct+0x70/0x70 [ 4484.281284] ret_from_fork+0x1f/0x30 [ 4498.079988] rcu: INFO: rcu_sched self-detected stall on CPU [ 4498.084917] rcu: 0-....: (1491245 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=451856 [ 4498.090961] (t=2400059 jiffies g=297885 q=12761) [ 4498.095559] NMI backtrace for cpu 0 [ 4498.098300] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4498.107463] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4498.114335] Call Trace: [ 4498.115626] [ 4498.116445] ? dump_stack+0xbb/0x10e [ 4498.119520] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4498.124282] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4498.129274] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4498.134853] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4498.140037] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4498.151029] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4498.153311] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4498.162132] ? update_process_times+0x6d/0xc0 [ 4498.166630] ? tick_sched_handle.isra.18+0x27/0x90 [ 4498.169190] ? tick_sched_timer+0x53/0xb0 [ 4498.177143] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4498.180697] ? tick_sched_handle.isra.18+0x90/0x90 [ 4498.183315] ? hrtimer_interrupt+0x128/0x2c0 [ 4498.186049] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4498.188213] ? apic_timer_interrupt+0xf/0x20 [ 4498.190209] [ 4498.191119] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4498.196271] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4498.198521] ? do_raw_spin_lock+0x177/0x190 [ 4498.200187] ? _raw_spin_lock+0x21/0x30 [ 4498.201638] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4498.206779] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4498.210098] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4498.212314] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4498.223242] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4498.237711] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4498.247175] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4498.259469] ? __next_timer_interrupt+0x160/0x160 [ 4498.261090] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4498.267118] ? do_wait_intr+0xf0/0xf0 [ 4498.268262] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4498.272466] ? kthread+0x1d1/0x200 [ 4498.273754] ? set_kthread_struct+0x70/0x70 [ 4498.277599] ? ret_from_fork+0x1f/0x30 [ 4504.059076] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4504.062693] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4504.104442] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4504.111934] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4504.119289] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4504.125420] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4504.139868] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4504.148019] RAX: 0000000000006c90 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4504.162213] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4504.174009] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4504.185413] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4504.195219] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4504.204427] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4504.213208] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4504.218600] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4504.223421] Call Trace: [ 4504.225454] [ 4504.226894] ? show_regs.cold.9+0x22/0x2f [ 4504.230521] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4504.235019] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4504.243908] ? watchdog+0x40/0x40 [ 4504.249876] ? hrtimer_interrupt+0x128/0x2c0 [ 4504.251483] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4504.252638] ? apic_timer_interrupt+0xf/0x20 [ 4504.254309] [ 4504.255114] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4504.257837] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4504.259919] do_raw_spin_lock+0x177/0x190 [ 4504.261597] _raw_spin_lock+0x21/0x30 [ 4504.263245] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4504.265732] ? do_raw_spin_unlock+0x75/0x190 [ 4504.267785] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4504.270273] ? do_wait_intr+0xf0/0xf0 [ 4504.272043] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4504.274339] mdc_reint+0x2d/0x110 [mdc] [ 4504.276060] mdc_unlink+0x658/0x870 [mdc] [ 4504.277944] lmv_unlink+0x34d/0xbd0 [lmv] [ 4504.279440] ll_unlink+0x24d/0x7f0 [lustre] [ 4504.280901] vfs_unlink+0x202/0x330 [ 4504.282448] do_unlinkat+0x2fa/0x460 [ 4504.283832] __x64_sys_unlink+0x37/0x50 [ 4504.285656] do_syscall_64+0xc1/0x440 [ 4504.287485] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4504.289455] RIP: 0033:0x7f33957fc10b [ 4504.290387] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4504.297435] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4504.300043] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4504.303166] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4504.306643] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4504.309860] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4504.312288] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4508.061391] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4508.072772] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4508.106975] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4508.125058] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4508.134912] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4508.139233] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4508.144365] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4508.156886] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4508.164112] RAX: 0000000000007e9c RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4508.168081] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4508.171671] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4508.175449] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4508.181056] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4508.185288] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4508.193724] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4508.198841] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4508.211552] Call Trace: [ 4508.213536] [ 4508.214664] ? show_regs.cold.9+0x22/0x2f [ 4508.218209] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4508.220701] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4508.223788] ? watchdog+0x40/0x40 [ 4508.225961] ? hrtimer_interrupt+0x128/0x2c0 [ 4508.228303] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4508.233856] ? apic_timer_interrupt+0xf/0x20 [ 4508.240569] [ 4508.243420] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4508.250489] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4508.254158] do_raw_spin_lock+0x177/0x190 [ 4508.255903] _raw_spin_lock+0x21/0x30 [ 4508.257812] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4508.261147] process_one_work+0x2c8/0x700 [ 4508.263426] worker_thread+0x296/0x6e0 [ 4508.265464] ? rescuer_thread+0x570/0x570 [ 4508.267705] kthread+0x1d1/0x200 [ 4508.269645] ? set_kthread_struct+0x70/0x70 [ 4508.272859] ret_from_fork+0x1f/0x30 [ 4524.063628] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4524.085587] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4524.145785] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4524.158458] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4524.164455] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4524.170478] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4524.199325] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4524.217537] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4524.224736] RDX: 00000000000022c0 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4524.232309] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4524.242277] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4524.257745] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4524.268427] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4524.280538] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4524.284039] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4524.289943] Call Trace: [ 4524.293957] [ 4524.295696] ? show_regs.cold.9+0x22/0x2f [ 4524.305744] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4524.309650] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4524.315713] ? watchdog+0x40/0x40 [ 4524.320280] ? hrtimer_interrupt+0x128/0x2c0 [ 4524.325049] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4524.332335] ? apic_timer_interrupt+0xf/0x20 [ 4524.334449] [ 4524.338479] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4524.340543] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4524.345279] do_raw_spin_lock+0x177/0x190 [ 4524.349418] _raw_spin_lock+0x21/0x30 [ 4524.356882] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4524.364940] obd_mod_free+0x46/0x140 [ptlrpc] [ 4524.369567] mdc_commit_open+0x91/0xb0 [mdc] [ 4524.374531] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4524.381219] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4524.390802] after_reply+0xe2c/0x1780 [ptlrpc] [ 4524.399316] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4524.409194] ? __next_timer_interrupt+0x160/0x160 [ 4524.414906] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4524.422654] ? do_wait_intr+0xf0/0xf0 [ 4524.427416] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4524.436475] kthread+0x1d1/0x200 [ 4524.443021] ? set_kthread_struct+0x70/0x70 [ 4524.448654] ret_from_fork+0x1f/0x30 [ 4532.059688] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4532.065776] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4532.123035] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4532.137904] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4532.150612] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4532.157117] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4532.186020] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4532.198328] RAX: 0000000000006b20 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4532.208696] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4532.221044] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4532.239434] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4532.251595] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4532.260366] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4532.270773] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4532.279402] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4532.284632] Call Trace: [ 4532.288641] [ 4532.290523] ? show_regs.cold.9+0x22/0x2f [ 4532.296956] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4532.302187] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4532.309594] ? watchdog+0x40/0x40 [ 4532.314504] ? hrtimer_interrupt+0x128/0x2c0 [ 4532.319822] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4532.325617] ? apic_timer_interrupt+0xf/0x20 [ 4532.333946] [ 4532.340992] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4532.347678] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4532.359823] do_raw_spin_lock+0x177/0x190 [ 4532.372538] _raw_spin_lock+0x21/0x30 [ 4532.374527] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4532.382341] ? do_raw_spin_unlock+0x75/0x190 [ 4532.389815] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4532.398164] ? do_wait_intr+0xf0/0xf0 [ 4532.400905] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4532.408245] mdc_reint+0x2d/0x110 [mdc] [ 4532.412008] mdc_unlink+0x658/0x870 [mdc] [ 4532.421921] lmv_unlink+0x34d/0xbd0 [lmv] [ 4532.427024] ll_unlink+0x24d/0x7f0 [lustre] [ 4532.430858] vfs_unlink+0x202/0x330 [ 4532.437728] do_unlinkat+0x2fa/0x460 [ 4532.439955] __x64_sys_unlink+0x37/0x50 [ 4532.448636] do_syscall_64+0xc1/0x440 [ 4532.456900] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4532.466692] RIP: 0033:0x7f33957fc10b [ 4532.473324] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4532.512885] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4532.531380] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4532.557874] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4532.572905] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4532.593407] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4532.602752] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4536.058699] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4536.069219] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4536.123042] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4536.138848] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4536.149920] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4536.154900] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4536.160897] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4536.182249] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4536.189499] RAX: 0000000000007f82 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4536.195250] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4536.202584] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4536.210091] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4536.215259] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4536.228426] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4536.236143] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4536.241060] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4536.245180] Call Trace: [ 4536.246850] [ 4536.248029] ? show_regs.cold.9+0x22/0x2f [ 4536.250613] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4536.253072] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4536.255129] ? watchdog+0x40/0x40 [ 4536.256795] ? hrtimer_interrupt+0x128/0x2c0 [ 4536.258714] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4536.261051] ? apic_timer_interrupt+0xf/0x20 [ 4536.262923] [ 4536.263979] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4536.266316] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4536.272553] do_raw_spin_lock+0x177/0x190 [ 4536.274650] _raw_spin_lock+0x21/0x30 [ 4536.276268] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4536.280951] process_one_work+0x2c8/0x700 [ 4536.282965] worker_thread+0x296/0x6e0 [ 4536.288367] ? rescuer_thread+0x570/0x570 [ 4536.298939] kthread+0x1d1/0x200 [ 4536.301245] ? set_kthread_struct+0x70/0x70 [ 4536.311932] ret_from_fork+0x1f/0x30 [ 4552.054052] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4552.067550] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4552.147832] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4552.170769] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4552.179250] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4552.182786] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4552.208082] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4552.221088] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4552.250641] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4552.270716] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4552.280724] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4552.300003] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4552.315455] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4552.334478] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4552.344159] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4552.359694] Call Trace: [ 4552.370293] [ 4552.375020] ? show_regs.cold.9+0x22/0x2f [ 4552.378377] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4552.383600] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4552.386208] ? watchdog+0x40/0x40 [ 4552.392567] ? hrtimer_interrupt+0x128/0x2c0 [ 4552.395178] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4552.397712] ? apic_timer_interrupt+0xf/0x20 [ 4552.402281] [ 4552.403230] ? native_safe_halt+0x2e/0x50 [ 4552.405582] kvm_wait+0xcd/0xe0 [ 4552.409664] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4552.413626] do_raw_spin_lock+0x177/0x190 [ 4552.416092] _raw_spin_lock+0x21/0x30 [ 4552.419214] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4552.422174] obd_mod_free+0x46/0x140 [ptlrpc] [ 4552.424818] mdc_commit_open+0x91/0xb0 [mdc] [ 4552.430209] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4552.433459] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4552.440921] after_reply+0xe2c/0x1780 [ptlrpc] [ 4552.450330] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4552.458865] ? __next_timer_interrupt+0x160/0x160 [ 4552.462521] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4552.467619] ? do_wait_intr+0xf0/0xf0 [ 4552.474409] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4552.484013] kthread+0x1d1/0x200 [ 4552.491053] ? set_kthread_struct+0x70/0x70 [ 4552.496831] ret_from_fork+0x1f/0x30 [ 4560.059134] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4560.065605] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4560.097004] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4560.104389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4560.110334] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4560.113306] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4560.124985] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4560.128996] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4560.133505] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4560.136234] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4560.141406] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4560.146122] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4560.150229] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4560.155342] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4560.158838] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4560.163587] Call Trace: [ 4560.165399] [ 4560.167627] ? show_regs.cold.9+0x22/0x2f [ 4560.170357] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4560.173708] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4560.176141] ? watchdog+0x40/0x40 [ 4560.178377] ? hrtimer_interrupt+0x128/0x2c0 [ 4560.180316] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4560.183192] ? apic_timer_interrupt+0xf/0x20 [ 4560.185806] [ 4560.187070] ? native_safe_halt+0x2e/0x50 [ 4560.190097] kvm_wait+0xcd/0xe0 [ 4560.193592] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4560.198829] do_raw_spin_lock+0x177/0x190 [ 4560.201026] _raw_spin_lock+0x21/0x30 [ 4560.206834] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4560.210906] ? do_raw_spin_unlock+0x75/0x190 [ 4560.213275] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4560.219464] ? do_wait_intr+0xf0/0xf0 [ 4560.222225] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4560.225979] mdc_reint+0x2d/0x110 [mdc] [ 4560.228033] mdc_unlink+0x658/0x870 [mdc] [ 4560.230496] lmv_unlink+0x34d/0xbd0 [lmv] [ 4560.232879] ll_unlink+0x24d/0x7f0 [lustre] [ 4560.235130] vfs_unlink+0x202/0x330 [ 4560.237003] do_unlinkat+0x2fa/0x460 [ 4560.242406] __x64_sys_unlink+0x37/0x50 [ 4560.245928] do_syscall_64+0xc1/0x440 [ 4560.247569] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4560.251058] RIP: 0033:0x7f33957fc10b [ 4560.253461] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4560.266007] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4560.270649] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4560.275458] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4560.279631] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4560.288009] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4560.295947] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4564.060952] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4564.070233] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4564.135449] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4564.148678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4564.154450] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4564.171883] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4564.193306] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4564.205990] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4564.211682] RAX: 0000000000002921 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4564.214893] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4564.218708] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4564.226476] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4564.229902] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4564.233446] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4564.239247] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4564.245670] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4564.248354] Call Trace: [ 4564.249025] [ 4564.259396] ? show_regs.cold.9+0x22/0x2f [ 4564.260734] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4564.264445] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4564.267839] ? watchdog+0x40/0x40 [ 4564.269495] ? hrtimer_interrupt+0x128/0x2c0 [ 4564.271844] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4564.274673] ? apic_timer_interrupt+0xf/0x20 [ 4564.277091] [ 4564.278092] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4564.281105] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4564.283618] do_raw_spin_lock+0x177/0x190 [ 4564.285539] _raw_spin_lock+0x21/0x30 [ 4564.287476] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4564.292358] process_one_work+0x2c8/0x700 [ 4564.297786] worker_thread+0x296/0x6e0 [ 4564.299599] ? rescuer_thread+0x570/0x570 [ 4564.301340] kthread+0x1d1/0x200 [ 4564.303135] ? set_kthread_struct+0x70/0x70 [ 4564.305962] ret_from_fork+0x1f/0x30 [ 4580.053097] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4580.062763] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4580.104932] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4580.114636] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4580.120468] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4580.123338] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4580.131832] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4580.136564] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4580.141187] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4580.148259] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4580.152549] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4580.156124] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4580.159205] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4580.162535] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4580.165152] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4580.173812] Call Trace: [ 4580.176616] [ 4580.179004] ? show_regs.cold.9+0x22/0x2f [ 4580.181000] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4580.184052] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4580.186177] ? watchdog+0x40/0x40 [ 4580.188332] ? hrtimer_interrupt+0x128/0x2c0 [ 4580.192732] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4580.198938] ? apic_timer_interrupt+0xf/0x20 [ 4580.202502] [ 4580.203502] ? native_safe_halt+0x2e/0x50 [ 4580.205616] kvm_wait+0xcd/0xe0 [ 4580.206928] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4580.209936] do_raw_spin_lock+0x177/0x190 [ 4580.212063] _raw_spin_lock+0x21/0x30 [ 4580.214218] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4580.216923] obd_mod_free+0x46/0x140 [ptlrpc] [ 4580.219493] mdc_commit_open+0x91/0xb0 [mdc] [ 4580.221621] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4580.224379] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4580.226967] after_reply+0xe2c/0x1780 [ptlrpc] [ 4580.230181] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4580.233892] ? __next_timer_interrupt+0x160/0x160 [ 4580.236659] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4580.239905] ? do_wait_intr+0xf0/0xf0 [ 4580.242101] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4580.245580] kthread+0x1d1/0x200 [ 4580.247842] ? set_kthread_struct+0x70/0x70 [ 4580.254060] ret_from_fork+0x1f/0x30 [ 4588.060872] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4588.066364] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4588.091027] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4588.096399] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4588.101453] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4588.105252] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4588.114408] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4588.118118] RAX: 0000000000005784 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4588.122508] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4588.126815] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4588.129952] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4588.134045] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4588.137330] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4588.140867] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4588.143900] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4588.147518] Call Trace: [ 4588.148696] [ 4588.149727] ? show_regs.cold.9+0x22/0x2f [ 4588.151212] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4588.153632] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4588.155647] ? watchdog+0x40/0x40 [ 4588.156747] ? hrtimer_interrupt+0x128/0x2c0 [ 4588.159153] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4588.160757] ? apic_timer_interrupt+0xf/0x20 [ 4588.162508] [ 4588.163529] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4588.165607] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4588.168193] do_raw_spin_lock+0x177/0x190 [ 4588.170042] _raw_spin_lock+0x21/0x30 [ 4588.171644] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4588.174076] ? do_raw_spin_unlock+0x75/0x190 [ 4588.175975] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4588.178351] ? do_wait_intr+0xf0/0xf0 [ 4588.179815] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4588.182356] mdc_reint+0x2d/0x110 [mdc] [ 4588.183976] mdc_unlink+0x658/0x870 [mdc] [ 4588.186213] lmv_unlink+0x34d/0xbd0 [lmv] [ 4588.187867] ll_unlink+0x24d/0x7f0 [lustre] [ 4588.189884] vfs_unlink+0x202/0x330 [ 4588.191372] do_unlinkat+0x2fa/0x460 [ 4588.192821] __x64_sys_unlink+0x37/0x50 [ 4588.194812] do_syscall_64+0xc1/0x440 [ 4588.196386] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4588.198725] RIP: 0033:0x7f33957fc10b [ 4588.200370] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4588.207678] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4588.210945] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4588.216745] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4588.220380] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4588.224228] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4588.227503] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4592.058022] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4592.065936] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4592.108569] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4592.126669] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4592.133352] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4592.136258] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4592.138913] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4592.150377] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4592.158882] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4592.166844] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4592.175859] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4592.178978] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4592.182259] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4592.185364] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4592.188689] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4592.193139] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4592.202172] Call Trace: [ 4592.205278] [ 4592.207650] ? show_regs.cold.9+0x22/0x2f [ 4592.215114] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4592.218641] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4592.219832] ? watchdog+0x40/0x40 [ 4592.227103] ? hrtimer_interrupt+0x128/0x2c0 [ 4592.228792] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4592.238130] ? apic_timer_interrupt+0xf/0x20 [ 4592.239456] [ 4592.240163] ? native_safe_halt+0x2e/0x50 [ 4592.244088] kvm_wait+0xcd/0xe0 [ 4592.248263] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4592.250623] do_raw_spin_lock+0x177/0x190 [ 4592.253320] _raw_spin_lock+0x21/0x30 [ 4592.255287] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4592.257033] process_one_work+0x2c8/0x700 [ 4592.263164] worker_thread+0x296/0x6e0 [ 4592.265200] ? rescuer_thread+0x570/0x570 [ 4592.267227] kthread+0x1d1/0x200 [ 4592.268261] ? set_kthread_struct+0x70/0x70 [ 4592.273314] ret_from_fork+0x1f/0x30 [ 4608.053178] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4608.059655] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4608.085969] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4608.090205] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4608.094116] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4608.096759] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4608.105223] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4608.108449] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4608.110873] RDX: 0000000000002cae RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4608.113273] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4608.115981] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4608.118659] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4608.121278] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4608.124222] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4608.126431] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4608.129538] Call Trace: [ 4608.130441] [ 4608.131130] ? show_regs.cold.9+0x22/0x2f [ 4608.132516] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4608.134551] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4608.136337] ? watchdog+0x40/0x40 [ 4608.137659] ? hrtimer_interrupt+0x128/0x2c0 [ 4608.139223] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4608.140833] ? apic_timer_interrupt+0xf/0x20 [ 4608.142507] [ 4608.143305] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4608.145260] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4608.147406] do_raw_spin_lock+0x177/0x190 [ 4608.148871] _raw_spin_lock+0x21/0x30 [ 4608.150051] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4608.152347] obd_mod_free+0x46/0x140 [ptlrpc] [ 4608.155484] mdc_commit_open+0x91/0xb0 [mdc] [ 4608.157691] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4608.162958] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4608.166284] after_reply+0xe2c/0x1780 [ptlrpc] [ 4608.168678] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4608.171785] ? __next_timer_interrupt+0x160/0x160 [ 4608.175872] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4608.179641] ? do_wait_intr+0xf0/0xf0 [ 4608.182658] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4608.186843] kthread+0x1d1/0x200 [ 4608.189859] ? set_kthread_struct+0x70/0x70 [ 4608.194908] ret_from_fork+0x1f/0x30 [ 4616.059085] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4616.063511] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4616.091978] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4616.101972] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4616.114336] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4616.119543] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4616.144304] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4616.156052] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4616.166794] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4616.186195] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4616.193031] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4616.199763] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4616.204973] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4616.213718] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4616.216513] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4616.224146] Call Trace: [ 4616.225157] [ 4616.227686] ? show_regs.cold.9+0x22/0x2f [ 4616.229757] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4616.232820] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4616.239347] ? watchdog+0x40/0x40 [ 4616.241839] ? hrtimer_interrupt+0x128/0x2c0 [ 4616.245211] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4616.247148] ? apic_timer_interrupt+0xf/0x20 [ 4616.251761] [ 4616.252542] ? native_safe_halt+0x2e/0x50 [ 4616.254370] kvm_wait+0xcd/0xe0 [ 4616.258521] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4616.266362] do_raw_spin_lock+0x177/0x190 [ 4616.267803] _raw_spin_lock+0x21/0x30 [ 4616.270104] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4616.275219] ? do_raw_spin_unlock+0x75/0x190 [ 4616.277938] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4616.284187] ? do_wait_intr+0xf0/0xf0 [ 4616.286682] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4616.290318] mdc_reint+0x2d/0x110 [mdc] [ 4616.292891] mdc_unlink+0x658/0x870 [mdc] [ 4616.295179] lmv_unlink+0x34d/0xbd0 [lmv] [ 4616.298259] ll_unlink+0x24d/0x7f0 [lustre] [ 4616.299735] vfs_unlink+0x202/0x330 [ 4616.306815] do_unlinkat+0x2fa/0x460 [ 4616.308171] __x64_sys_unlink+0x37/0x50 [ 4616.317083] do_syscall_64+0xc1/0x440 [ 4616.318531] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4616.324507] RIP: 0033:0x7f33957fc10b [ 4616.329757] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4616.346793] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4616.353738] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4616.359917] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4616.366864] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4616.373315] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4616.378858] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4620.059789] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4620.066296] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4620.107806] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4620.115772] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4620.120643] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4620.124294] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4620.126817] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4620.139994] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4620.142465] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4620.145742] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4620.152471] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4620.163811] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4620.171739] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4620.178810] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4620.187773] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4620.191075] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4620.197318] Call Trace: [ 4620.203371] [ 4620.204608] ? show_regs.cold.9+0x22/0x2f [ 4620.210076] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4620.212501] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4620.215559] ? watchdog+0x40/0x40 [ 4620.217442] ? hrtimer_interrupt+0x128/0x2c0 [ 4620.219803] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4620.222577] ? apic_timer_interrupt+0xf/0x20 [ 4620.224757] [ 4620.226293] ? native_safe_halt+0x2e/0x50 [ 4620.228540] kvm_wait+0xcd/0xe0 [ 4620.230117] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4620.232663] do_raw_spin_lock+0x177/0x190 [ 4620.234768] _raw_spin_lock+0x21/0x30 [ 4620.240241] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4620.245929] process_one_work+0x2c8/0x700 [ 4620.251192] worker_thread+0x296/0x6e0 [ 4620.256360] ? rescuer_thread+0x570/0x570 [ 4620.258105] kthread+0x1d1/0x200 [ 4620.262517] ? set_kthread_struct+0x70/0x70 [ 4620.265222] ret_from_fork+0x1f/0x30 [ 4636.055715] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4636.063740] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4636.122262] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4636.128539] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4636.140758] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4636.143905] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4636.163450] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4636.173236] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4636.184044] RDX: 0000000000004886 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4636.197387] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4636.209403] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4636.215455] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4636.221885] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4636.229335] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4636.235265] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4636.248373] Call Trace: [ 4636.252618] [ 4636.255285] ? show_regs.cold.9+0x22/0x2f [ 4636.263385] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4636.269319] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4636.273062] ? watchdog+0x40/0x40 [ 4636.277656] ? hrtimer_interrupt+0x128/0x2c0 [ 4636.283145] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4636.286254] ? apic_timer_interrupt+0xf/0x20 [ 4636.292125] [ 4636.292957] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4636.298243] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4636.306766] do_raw_spin_lock+0x177/0x190 [ 4636.312360] _raw_spin_lock+0x21/0x30 [ 4636.315267] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4636.321172] obd_mod_free+0x46/0x140 [ptlrpc] [ 4636.328385] mdc_commit_open+0x91/0xb0 [mdc] [ 4636.330716] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4636.333780] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4636.339156] after_reply+0xe2c/0x1780 [ptlrpc] [ 4636.343874] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4636.348833] ? __next_timer_interrupt+0x160/0x160 [ 4636.352142] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4636.356250] ? do_wait_intr+0xf0/0xf0 [ 4636.358379] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4636.362071] kthread+0x1d1/0x200 [ 4636.364472] ? set_kthread_struct+0x70/0x70 [ 4636.369109] ret_from_fork+0x1f/0x30 [ 4644.059030] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4644.062237] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4644.101980] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4644.107761] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4644.112054] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4644.114097] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4644.132727] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4644.136919] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4644.140582] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4644.143988] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4644.146679] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4644.150587] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4644.154344] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4644.158382] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4644.161186] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4644.165298] Call Trace: [ 4644.167255] [ 4644.168164] ? show_regs.cold.9+0x22/0x2f [ 4644.170927] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4644.173222] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4644.176088] ? watchdog+0x40/0x40 [ 4644.177263] ? hrtimer_interrupt+0x128/0x2c0 [ 4644.179530] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4644.181976] ? apic_timer_interrupt+0xf/0x20 [ 4644.185006] [ 4644.186210] ? native_safe_halt+0x2e/0x50 [ 4644.188157] kvm_wait+0xcd/0xe0 [ 4644.189612] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4644.191702] do_raw_spin_lock+0x177/0x190 [ 4644.193902] _raw_spin_lock+0x21/0x30 [ 4644.196142] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4644.206036] ? do_raw_spin_unlock+0x75/0x190 [ 4644.210958] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4644.215207] ? do_wait_intr+0xf0/0xf0 [ 4644.217157] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4644.220290] mdc_reint+0x2d/0x110 [mdc] [ 4644.222874] mdc_unlink+0x658/0x870 [mdc] [ 4644.224832] lmv_unlink+0x34d/0xbd0 [lmv] [ 4644.228085] ll_unlink+0x24d/0x7f0 [lustre] [ 4644.230508] vfs_unlink+0x202/0x330 [ 4644.232751] do_unlinkat+0x2fa/0x460 [ 4644.234417] __x64_sys_unlink+0x37/0x50 [ 4644.239241] do_syscall_64+0xc1/0x440 [ 4644.241978] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4644.247341] RIP: 0033:0x7f33957fc10b [ 4644.249917] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4644.262666] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4644.266128] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4644.270388] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4644.274355] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4644.278798] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4644.282296] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4648.058379] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4648.062874] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4648.100812] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4648.117649] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4648.130234] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4648.146633] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4648.159931] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4648.177221] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4648.180626] RAX: 00000000000060a5 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4648.188028] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4648.203837] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4648.211849] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4648.221214] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4648.227829] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4648.232057] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4648.237333] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4648.241152] Call Trace: [ 4648.242334] [ 4648.243279] ? show_regs.cold.9+0x22/0x2f [ 4648.246204] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4648.248097] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4648.249994] ? watchdog+0x40/0x40 [ 4648.251606] ? hrtimer_interrupt+0x128/0x2c0 [ 4648.254273] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4648.256670] ? apic_timer_interrupt+0xf/0x20 [ 4648.258822] [ 4648.264410] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4648.269437] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4648.273750] do_raw_spin_lock+0x177/0x190 [ 4648.278110] _raw_spin_lock+0x21/0x30 [ 4648.279805] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4648.282780] process_one_work+0x2c8/0x700 [ 4648.285238] worker_thread+0x296/0x6e0 [ 4648.287109] ? rescuer_thread+0x570/0x570 [ 4648.289862] kthread+0x1d1/0x200 [ 4648.292965] ? set_kthread_struct+0x70/0x70 [ 4648.295637] ret_from_fork+0x1f/0x30 [ 4664.053027] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4664.063803] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4664.101212] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4664.112367] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4664.116679] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4664.118827] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4664.134681] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4664.138309] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4664.142842] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4664.146412] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4664.150118] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4664.153982] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4664.157925] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4664.162118] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4664.164965] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4664.168786] Call Trace: [ 4664.170120] [ 4664.171288] ? show_regs.cold.9+0x22/0x2f [ 4664.173150] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4664.175645] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4664.178589] ? watchdog+0x40/0x40 [ 4664.180401] ? hrtimer_interrupt+0x128/0x2c0 [ 4664.182332] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4664.184512] ? apic_timer_interrupt+0xf/0x20 [ 4664.186835] [ 4664.188477] ? native_safe_halt+0x2e/0x50 [ 4664.190398] kvm_wait+0xcd/0xe0 [ 4664.191761] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4664.194091] do_raw_spin_lock+0x177/0x190 [ 4664.196270] _raw_spin_lock+0x21/0x30 [ 4664.198251] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4664.200765] obd_mod_free+0x46/0x140 [ptlrpc] [ 4664.203653] mdc_commit_open+0x91/0xb0 [mdc] [ 4664.205724] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4664.208754] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4664.213309] after_reply+0xe2c/0x1780 [ptlrpc] [ 4664.216096] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4664.220962] ? __next_timer_interrupt+0x160/0x160 [ 4664.224278] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4664.229663] ? do_wait_intr+0xf0/0xf0 [ 4664.240229] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4664.248465] kthread+0x1d1/0x200 [ 4664.252583] ? set_kthread_struct+0x70/0x70 [ 4664.256563] ret_from_fork+0x1f/0x30 [ 4672.059027] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4672.063072] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4672.103759] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4672.110814] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4672.114795] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4672.116403] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4672.128514] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4672.131846] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4672.134632] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4672.139406] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4672.142643] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4672.149275] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4672.152498] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4672.161717] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4672.166471] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4672.170423] Call Trace: [ 4672.171404] [ 4672.172321] ? show_regs.cold.9+0x22/0x2f [ 4672.173704] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4672.175671] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4672.177479] ? watchdog+0x40/0x40 [ 4672.179306] ? hrtimer_interrupt+0x128/0x2c0 [ 4672.180672] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4672.182749] ? apic_timer_interrupt+0xf/0x20 [ 4672.184287] [ 4672.185126] ? native_safe_halt+0x2e/0x50 [ 4672.186718] kvm_wait+0xcd/0xe0 [ 4672.188061] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4672.189672] do_raw_spin_lock+0x177/0x190 [ 4672.191043] _raw_spin_lock+0x21/0x30 [ 4672.192369] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4672.194415] ? do_raw_spin_unlock+0x75/0x190 [ 4672.195719] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4672.197745] ? do_wait_intr+0xf0/0xf0 [ 4672.198991] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4672.201096] mdc_reint+0x2d/0x110 [mdc] [ 4672.202426] mdc_unlink+0x658/0x870 [mdc] [ 4672.204060] lmv_unlink+0x34d/0xbd0 [lmv] [ 4672.206684] ll_unlink+0x24d/0x7f0 [lustre] [ 4672.208930] vfs_unlink+0x202/0x330 [ 4672.210638] do_unlinkat+0x2fa/0x460 [ 4672.212072] __x64_sys_unlink+0x37/0x50 [ 4672.213497] do_syscall_64+0xc1/0x440 [ 4672.215392] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4672.217467] RIP: 0033:0x7f33957fc10b [ 4672.218807] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4672.227105] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4672.233628] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4672.242555] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4672.249677] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4672.252231] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4672.255871] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4676.063883] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4676.069200] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4676.157906] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4676.170335] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4676.180703] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4676.188178] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4676.199416] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4676.219717] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4676.229465] RAX: 0000000000007e34 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4676.236854] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4676.240672] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4676.249234] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4676.255159] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4676.267156] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4676.277691] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4676.283572] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4676.292672] Call Trace: [ 4676.295457] [ 4676.298883] ? show_regs.cold.9+0x22/0x2f [ 4676.303847] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4676.307753] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4676.312937] ? watchdog+0x40/0x40 [ 4676.316299] ? hrtimer_interrupt+0x128/0x2c0 [ 4676.319625] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4676.323438] ? apic_timer_interrupt+0xf/0x20 [ 4676.325352] [ 4676.326331] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4676.328709] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4676.331246] do_raw_spin_lock+0x177/0x190 [ 4676.332931] _raw_spin_lock+0x21/0x30 [ 4676.334708] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4676.342282] process_one_work+0x2c8/0x700 [ 4676.344881] worker_thread+0x296/0x6e0 [ 4676.349941] ? rescuer_thread+0x570/0x570 [ 4676.358248] kthread+0x1d1/0x200 [ 4676.362346] ? set_kthread_struct+0x70/0x70 [ 4676.367479] ret_from_fork+0x1f/0x30 [ 4678.083128] rcu: INFO: rcu_sched self-detected stall on CPU [ 4678.087096] rcu: 0-....: (1600986 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=485339 [ 4678.092534] (t=2580063 jiffies g=297885 q=13459) [ 4678.100955] NMI backtrace for cpu 0 [ 4678.102335] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4678.109638] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4678.114077] Call Trace: [ 4678.115615] [ 4678.116665] ? dump_stack+0xbb/0x10e [ 4678.122277] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4678.124415] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4678.128378] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4678.131838] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4678.133876] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4678.141309] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4678.148347] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4678.150967] ? update_process_times+0x6d/0xc0 [ 4678.155685] ? tick_sched_handle.isra.18+0x27/0x90 [ 4678.159958] ? tick_sched_timer+0x53/0xb0 [ 4678.162275] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4678.164395] ? tick_sched_handle.isra.18+0x90/0x90 [ 4678.168326] ? hrtimer_interrupt+0x128/0x2c0 [ 4678.169490] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4678.170716] ? apic_timer_interrupt+0xf/0x20 [ 4678.172657] [ 4678.173828] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4678.179242] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4678.181268] ? do_raw_spin_lock+0x177/0x190 [ 4678.183027] ? _raw_spin_lock+0x21/0x30 [ 4678.184589] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4678.190370] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4678.194152] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4678.196623] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4678.204461] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4678.213541] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4678.215347] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4678.220571] ? __next_timer_interrupt+0x160/0x160 [ 4678.222023] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4678.226095] ? do_wait_intr+0xf0/0xf0 [ 4678.227246] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4678.232354] ? kthread+0x1d1/0x200 [ 4678.233641] ? set_kthread_struct+0x70/0x70 [ 4678.235226] ? ret_from_fork+0x1f/0x30 [ 4700.059373] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4700.064772] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4700.084719] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4700.090172] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4700.102254] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4700.107244] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4700.116314] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4700.120119] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4700.127770] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4700.131944] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4700.137543] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4700.140761] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4700.146220] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4700.150227] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4700.154814] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4700.161010] Call Trace: [ 4700.168108] [ 4700.170498] ? show_regs.cold.9+0x22/0x2f [ 4700.173019] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4700.175157] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4700.177623] ? watchdog+0x40/0x40 [ 4700.179528] ? hrtimer_interrupt+0x128/0x2c0 [ 4700.182797] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4700.189206] ? apic_timer_interrupt+0xf/0x20 [ 4700.194108] [ 4700.195028] ? native_safe_halt+0x2e/0x50 [ 4700.196489] kvm_wait+0xcd/0xe0 [ 4700.200011] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4700.204054] do_raw_spin_lock+0x177/0x190 [ 4700.207180] _raw_spin_lock+0x21/0x30 [ 4700.210148] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4700.212623] ? do_raw_spin_unlock+0x75/0x190 [ 4700.216246] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4700.220955] ? do_wait_intr+0xf0/0xf0 [ 4700.224907] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4700.234302] mdc_reint+0x2d/0x110 [mdc] [ 4700.237949] mdc_unlink+0x658/0x870 [mdc] [ 4700.240100] lmv_unlink+0x34d/0xbd0 [lmv] [ 4700.243885] ll_unlink+0x24d/0x7f0 [lustre] [ 4700.247443] vfs_unlink+0x202/0x330 [ 4700.249932] do_unlinkat+0x2fa/0x460 [ 4700.253025] __x64_sys_unlink+0x37/0x50 [ 4700.264197] do_syscall_64+0xc1/0x440 [ 4700.265607] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4700.274516] RIP: 0033:0x7f33957fc10b [ 4700.275590] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4700.296094] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4700.307277] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4700.322936] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4700.328079] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4700.333254] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4700.338370] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4704.060358] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4704.064397] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4704.080677] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) [ 4704.092311] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4704.092395] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4704.092406] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4704.092415] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4704.093137] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4704.093148] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4704.093152] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 4704.093156] RAX: 000000000000782a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4704.093159] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4704.093161] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4704.093163] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4704.093165] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4704.093168] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4704.093171] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4704.093172] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4704.093180] Call Trace: [ 4704.093187] [ 4704.093194] ? show_regs.cold.9+0x22/0x2f [ 4704.093206] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4704.093212] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4704.093217] ? watchdog+0x40/0x40 [ 4704.093221] ? hrtimer_interrupt+0x128/0x2c0 [ 4704.093224] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4704.093234] ? apic_timer_interrupt+0xf/0x20 [ 4704.093239] [ 4704.093241] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4704.093245] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4704.093251] do_raw_spin_lock+0x177/0x190 [ 4704.093256] _raw_spin_lock+0x21/0x30 [ 4704.093262] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4704.093717] process_one_work+0x2c8/0x700 [ 4704.093726] worker_thread+0x296/0x6e0 [ 4704.093731] ? rescuer_thread+0x570/0x570 [ 4704.093735] kthread+0x1d1/0x200 [ 4704.093740] ? set_kthread_struct+0x70/0x70 [ 4704.093743] ret_from_fork+0x1f/0x30 [ 4704.299704] ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4704.324284] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4704.330595] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4704.338203] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4704.344750] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4704.362760] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4704.370438] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4704.377377] RDX: 000000000000305e RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4704.380760] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4704.385873] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4704.392779] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4704.402519] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4704.412394] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4704.415460] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4704.419419] Call Trace: [ 4704.421875] [ 4704.423170] ? show_regs.cold.9+0x22/0x2f [ 4704.428052] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4704.432286] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4704.437708] ? watchdog+0x40/0x40 [ 4704.439791] ? hrtimer_interrupt+0x128/0x2c0 [ 4704.441732] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4704.446403] ? apic_timer_interrupt+0xf/0x20 [ 4704.454069] [ 4704.455131] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4704.458738] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4704.462134] do_raw_spin_lock+0x177/0x190 [ 4704.468200] _raw_spin_lock+0x21/0x30 [ 4704.474485] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4704.479340] obd_mod_free+0x46/0x140 [ptlrpc] [ 4704.486961] mdc_commit_open+0x91/0xb0 [mdc] [ 4704.490473] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4704.495382] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4704.500667] after_reply+0xe2c/0x1780 [ptlrpc] [ 4704.503962] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4704.509763] ? __next_timer_interrupt+0x160/0x160 [ 4704.515858] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4704.519085] ? do_wait_intr+0xf0/0xf0 [ 4704.522972] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4704.526916] kthread+0x1d1/0x200 [ 4704.528768] ? set_kthread_struct+0x70/0x70 [ 4704.531142] ret_from_fork+0x1f/0x30 [ 4728.059026] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4728.079445] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4728.126254] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4728.142528] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4728.150026] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4728.159998] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4728.182504] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4728.191945] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4728.197470] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4728.208515] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4728.213393] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4728.218931] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4728.221083] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4728.227686] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4728.233554] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4728.236619] Call Trace: [ 4728.237388] [ 4728.238078] ? show_regs.cold.9+0x22/0x2f [ 4728.240878] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4728.244014] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4728.246229] ? watchdog+0x40/0x40 [ 4728.247357] ? hrtimer_interrupt+0x128/0x2c0 [ 4728.249104] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4728.250995] ? apic_timer_interrupt+0xf/0x20 [ 4728.253391] [ 4728.255689] ? native_safe_halt+0x2e/0x50 [ 4728.257111] kvm_wait+0xcd/0xe0 [ 4728.258309] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4728.260121] do_raw_spin_lock+0x177/0x190 [ 4728.261567] _raw_spin_lock+0x21/0x30 [ 4728.262710] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4728.264917] ? do_raw_spin_unlock+0x75/0x190 [ 4728.266943] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4728.277487] ? do_wait_intr+0xf0/0xf0 [ 4728.282625] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4728.294292] mdc_reint+0x2d/0x110 [mdc] [ 4728.298747] mdc_unlink+0x658/0x870 [mdc] [ 4728.299953] lmv_unlink+0x34d/0xbd0 [lmv] [ 4728.309732] ll_unlink+0x24d/0x7f0 [lustre] [ 4728.311015] vfs_unlink+0x202/0x330 [ 4728.318633] do_unlinkat+0x2fa/0x460 [ 4728.319786] __x64_sys_unlink+0x37/0x50 [ 4728.320837] do_syscall_64+0xc1/0x440 [ 4728.331090] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4728.332694] RIP: 0033:0x7f33957fc10b [ 4728.345121] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4728.361840] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4728.365660] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4728.369602] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4728.372356] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4728.375347] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4728.385707] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4732.053249] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 4732.058046] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4732.061183] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4732.061266] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4732.061274] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4732.061278] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4732.061298] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4732.061303] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4732.061308] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4732.061310] RDX: 000000000000783e RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4732.061313] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4732.061315] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4732.061317] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4732.061320] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4732.061322] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4732.061324] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4732.061333] Call Trace: [ 4732.061338] [ 4732.061345] ? show_regs.cold.9+0x22/0x2f [ 4732.061356] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4732.061363] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4732.061367] ? watchdog+0x40/0x40 [ 4732.061371] ? hrtimer_interrupt+0x128/0x2c0 [ 4732.061374] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4732.061383] ? apic_timer_interrupt+0xf/0x20 [ 4732.061387] [ 4732.061389] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4732.061394] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4732.061398] do_raw_spin_lock+0x177/0x190 [ 4732.061403] _raw_spin_lock+0x21/0x30 [ 4732.061408] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4732.068397] Modules linked in: lustre(O) [ 4732.107561] obd_mod_free+0x46/0x140 [ptlrpc] [ 4732.108298] mdc_commit_open+0x91/0xb0 [mdc] [ 4732.108354] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4732.108563] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4732.108878] after_reply+0xe2c/0x1780 [ptlrpc] [ 4732.109099] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4732.109301] ? __next_timer_interrupt+0x160/0x160 [ 4732.109312] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4732.109517] ? do_wait_intr+0xf0/0xf0 [ 4732.109526] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4732.109899] kthread+0x1d1/0x200 [ 4732.109905] ? set_kthread_struct+0x70/0x70 [ 4732.110066] ret_from_fork+0x1f/0x30 [ 4732.224871] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4732.257629] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4732.264305] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4732.270686] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4732.274979] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4732.277967] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4732.301342] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4732.308413] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4732.315852] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4732.321616] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4732.326912] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4732.330526] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4732.334351] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4732.340466] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4732.345597] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4732.351838] Call Trace: [ 4732.353423] [ 4732.354885] ? show_regs.cold.9+0x22/0x2f [ 4732.357631] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4732.361973] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4732.368541] ? watchdog+0x40/0x40 [ 4732.375357] ? hrtimer_interrupt+0x128/0x2c0 [ 4732.381025] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4732.386874] ? apic_timer_interrupt+0xf/0x20 [ 4732.389227] [ 4732.390418] ? native_safe_halt+0x2e/0x50 [ 4732.392469] kvm_wait+0xcd/0xe0 [ 4732.397298] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4732.401526] do_raw_spin_lock+0x177/0x190 [ 4732.406932] _raw_spin_lock+0x21/0x30 [ 4732.410274] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4732.414757] process_one_work+0x2c8/0x700 [ 4732.417166] worker_thread+0x296/0x6e0 [ 4732.419405] ? rescuer_thread+0x570/0x570 [ 4732.424392] kthread+0x1d1/0x200 [ 4732.428371] ? set_kthread_struct+0x70/0x70 [ 4732.435368] ret_from_fork+0x1f/0x30 [ 4756.059880] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4756.070902] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4756.119686] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4756.124804] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4756.129509] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4756.132371] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4756.142596] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4756.146047] RAX: 0000000000007f56 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4756.149349] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4756.152736] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4756.156948] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4756.160061] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4756.163044] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4756.165951] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4756.167975] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4756.172744] Call Trace: [ 4756.174987] [ 4756.178710] ? show_regs.cold.9+0x22/0x2f [ 4756.182021] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4756.186585] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4756.189564] ? watchdog+0x40/0x40 [ 4756.190832] ? hrtimer_interrupt+0x128/0x2c0 [ 4756.196492] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4756.199702] ? apic_timer_interrupt+0xf/0x20 [ 4756.204063] [ 4756.205547] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4756.209525] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4756.213081] do_raw_spin_lock+0x177/0x190 [ 4756.215899] _raw_spin_lock+0x21/0x30 [ 4756.218426] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4756.223940] ? do_raw_spin_unlock+0x75/0x190 [ 4756.226857] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4756.231087] ? do_wait_intr+0xf0/0xf0 [ 4756.233807] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4756.236849] mdc_reint+0x2d/0x110 [mdc] [ 4756.240097] mdc_unlink+0x658/0x870 [mdc] [ 4756.242619] lmv_unlink+0x34d/0xbd0 [lmv] [ 4756.244690] ll_unlink+0x24d/0x7f0 [lustre] [ 4756.247111] vfs_unlink+0x202/0x330 [ 4756.249034] do_unlinkat+0x2fa/0x460 [ 4756.250149] __x64_sys_unlink+0x37/0x50 [ 4756.255593] do_syscall_64+0xc1/0x440 [ 4756.259280] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4756.262745] RIP: 0033:0x7f33957fc10b [ 4756.264340] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4756.273593] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4756.277754] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4756.283641] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4756.288582] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4756.292780] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4756.296697] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4760.053063] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 4760.059076] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4760.064509] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4760.116431] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4760.116445] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4760.116449] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4760.116464] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4760.119558] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4760.119640] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4760.119645] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4760.119652] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4760.120704] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4760.120716] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4760.120721] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4760.120791] RAX: 0000000000007e08 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4760.120793] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4760.120795] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4760.120797] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4760.120799] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4760.120802] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4760.120804] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4760.120806] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4760.120814] Call Trace: [ 4760.120819] [ 4760.120824] ? show_regs.cold.9+0x22/0x2f [ 4760.120833] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4760.120839] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4760.120843] ? watchdog+0x40/0x40 [ 4760.120846] ? hrtimer_interrupt+0x128/0x2c0 [ 4760.120850] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4760.120858] ? apic_timer_interrupt+0xf/0x20 [ 4760.120862] [ 4760.120864] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4760.120869] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4760.120874] do_raw_spin_lock+0x177/0x190 [ 4760.120879] _raw_spin_lock+0x21/0x30 [ 4760.120891] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4760.140880] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4760.140897] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4760.140900] RDX: 0000000000000b29 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4760.140902] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4760.140904] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4760.140906] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4760.140909] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4760.140911] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4760.140913] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4760.140922] Call Trace: [ 4760.140929] [ 4760.140935] ? show_regs.cold.9+0x22/0x2f [ 4760.140947] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4760.140954] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4760.140959] ? watchdog+0x40/0x40 [ 4760.140963] ? hrtimer_interrupt+0x128/0x2c0 [ 4760.140967] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4760.140975] ? apic_timer_interrupt+0xf/0x20 [ 4760.140979] [ 4760.140982] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4760.140989] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4760.140993] do_raw_spin_lock+0x177/0x190 [ 4760.140997] _raw_spin_lock+0x21/0x30 [ 4760.141003] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4760.148687] process_one_work+0x2c8/0x700 [ 4760.148706] worker_thread+0x296/0x6e0 [ 4760.148712] ? rescuer_thread+0x570/0x570 [ 4760.148716] kthread+0x1d1/0x200 [ 4760.148721] ? set_kthread_struct+0x70/0x70 [ 4760.148725] ret_from_fork+0x1f/0x30 [ 4760.561012] obd_mod_free+0x46/0x140 [ptlrpc] [ 4760.564082] mdc_commit_open+0x91/0xb0 [mdc] [ 4760.566516] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4760.569170] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4760.573765] after_reply+0xe2c/0x1780 [ptlrpc] [ 4760.580378] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4760.586172] ? __next_timer_interrupt+0x160/0x160 [ 4760.591985] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4760.602860] ? do_wait_intr+0xf0/0xf0 [ 4760.607397] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4760.611277] kthread+0x1d1/0x200 [ 4760.613992] ? set_kthread_struct+0x70/0x70 [ 4760.616725] ret_from_fork+0x1f/0x30 [ 4784.059013] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4784.063579] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4784.123589] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4784.138615] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4784.146204] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4784.155588] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4784.172185] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4784.181935] RAX: 0000000000005608 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4784.185577] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4784.190176] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4784.199478] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4784.208080] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4784.211173] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4784.217914] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4784.221401] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4784.225822] Call Trace: [ 4784.229425] [ 4784.230583] ? show_regs.cold.9+0x22/0x2f [ 4784.232426] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4784.234398] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4784.236929] ? watchdog+0x40/0x40 [ 4784.238444] ? hrtimer_interrupt+0x128/0x2c0 [ 4784.240455] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4784.242833] ? apic_timer_interrupt+0xf/0x20 [ 4784.245001] [ 4784.245876] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4784.247877] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4784.250181] do_raw_spin_lock+0x177/0x190 [ 4784.251998] _raw_spin_lock+0x21/0x30 [ 4784.253796] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4784.256384] ? do_raw_spin_unlock+0x75/0x190 [ 4784.258309] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4784.260820] ? do_wait_intr+0xf0/0xf0 [ 4784.262522] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4784.265402] mdc_reint+0x2d/0x110 [mdc] [ 4784.266896] mdc_unlink+0x658/0x870 [mdc] [ 4784.268144] lmv_unlink+0x34d/0xbd0 [lmv] [ 4784.270107] ll_unlink+0x24d/0x7f0 [lustre] [ 4784.272347] vfs_unlink+0x202/0x330 [ 4784.274000] do_unlinkat+0x2fa/0x460 [ 4784.275761] __x64_sys_unlink+0x37/0x50 [ 4784.277563] do_syscall_64+0xc1/0x440 [ 4784.279396] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4784.281932] RIP: 0033:0x7f33957fc10b [ 4784.283616] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4784.292344] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4784.296610] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4784.302327] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4784.305503] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4784.308351] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4784.311497] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4788.053458] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 4788.058187] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4788.058428] Modules linked in: lustre(O) osp(O) [ 4788.063782] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4788.063867] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4788.063874] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4788.063880] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4788.064221] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4788.064231] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4788.064235] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4788.064239] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4788.064241] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4788.064243] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4788.064246] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4788.064248] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4788.064251] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4788.064253] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4788.064256] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4788.064264] Call Trace: [ 4788.064270] [ 4788.064276] ? show_regs.cold.9+0x22/0x2f [ 4788.064285] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4788.064291] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4788.064295] ? watchdog+0x40/0x40 [ 4788.064299] ? hrtimer_interrupt+0x128/0x2c0 [ 4788.064302] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4788.064309] ? apic_timer_interrupt+0xf/0x20 [ 4788.064313] [ 4788.064315] ? native_safe_halt+0x2e/0x50 [ 4788.064319] kvm_wait+0xcd/0xe0 [ 4788.064326] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4788.064334] do_raw_spin_lock+0x177/0x190 [ 4788.064339] _raw_spin_lock+0x21/0x30 [ 4788.064343] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4788.064545] process_one_work+0x2c8/0x700 [ 4788.064552] worker_thread+0x296/0x6e0 [ 4788.064556] ? rescuer_thread+0x570/0x570 [ 4788.064559] kthread+0x1d1/0x200 [ 4788.064563] ? set_kthread_struct+0x70/0x70 [ 4788.064566] ret_from_fork+0x1f/0x30 [ 4788.200098] ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4788.219260] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4788.224071] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4788.228377] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4788.230825] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4788.239056] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4788.242186] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4788.245343] RDX: 0000000000005542 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4788.248271] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4788.251297] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4788.254361] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4788.257595] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4788.261270] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4788.263917] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4788.267275] Call Trace: [ 4788.268311] [ 4788.269319] ? show_regs.cold.9+0x22/0x2f [ 4788.271189] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4788.273177] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4788.275199] ? watchdog+0x40/0x40 [ 4788.276866] ? hrtimer_interrupt+0x128/0x2c0 [ 4788.278810] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4788.281191] ? apic_timer_interrupt+0xf/0x20 [ 4788.283006] [ 4788.283824] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4788.286416] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4788.288849] do_raw_spin_lock+0x177/0x190 [ 4788.290646] _raw_spin_lock+0x21/0x30 [ 4788.292239] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4788.294955] obd_mod_free+0x46/0x140 [ptlrpc] [ 4788.297187] mdc_commit_open+0x91/0xb0 [mdc] [ 4788.299954] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4788.302942] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4788.305326] after_reply+0xe2c/0x1780 [ptlrpc] [ 4788.307714] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4788.310226] ? __next_timer_interrupt+0x160/0x160 [ 4788.312239] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4788.314408] ? do_wait_intr+0xf0/0xf0 [ 4788.316435] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4788.319192] kthread+0x1d1/0x200 [ 4788.320708] ? set_kthread_struct+0x70/0x70 [ 4788.322675] ret_from_fork+0x1f/0x30 [ 4812.062653] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4812.067864] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4812.097002] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4812.111502] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4812.117030] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4812.121580] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4812.136741] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4812.142090] RAX: 0000000000007b8e RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4812.146701] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4812.150899] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4812.155849] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4812.166980] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4812.171077] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4812.176977] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4812.180014] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4812.183726] Call Trace: [ 4812.186529] [ 4812.187397] ? show_regs.cold.9+0x22/0x2f [ 4812.190843] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4812.192820] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4812.194936] ? watchdog+0x40/0x40 [ 4812.202392] ? hrtimer_interrupt+0x128/0x2c0 [ 4812.204543] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4812.206339] ? apic_timer_interrupt+0xf/0x20 [ 4812.208644] [ 4812.210073] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4812.213048] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4812.216342] do_raw_spin_lock+0x177/0x190 [ 4812.225458] _raw_spin_lock+0x21/0x30 [ 4812.227110] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4812.230325] ? do_raw_spin_unlock+0x75/0x190 [ 4812.232557] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4812.235643] ? do_wait_intr+0xf0/0xf0 [ 4812.237620] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4812.239990] mdc_reint+0x2d/0x110 [mdc] [ 4812.243332] mdc_unlink+0x658/0x870 [mdc] [ 4812.244805] lmv_unlink+0x34d/0xbd0 [lmv] [ 4812.247331] ll_unlink+0x24d/0x7f0 [lustre] [ 4812.251212] vfs_unlink+0x202/0x330 [ 4812.252755] do_unlinkat+0x2fa/0x460 [ 4812.254766] __x64_sys_unlink+0x37/0x50 [ 4812.256568] do_syscall_64+0xc1/0x440 [ 4812.257991] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4812.260751] RIP: 0033:0x7f33957fc10b [ 4812.262287] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4812.281600] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4812.287598] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4812.292649] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4812.309173] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4812.312606] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4812.319567] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4816.055228] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 4816.062780] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 4816.074640] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4816.074836] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4816.074842] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4816.074844] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4816.074858] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4816.074862] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4816.074867] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4816.074869] RDX: 0000000000007e7d RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4816.074871] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4816.074873] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4816.074875] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4816.074878] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4816.074880] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4816.074882] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4816.075351] Call Trace: [ 4816.080342] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4816.080429] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4816.080435] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4816.080442] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4816.080834] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4816.080844] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4816.080848] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4816.080853] RAX: 0000000000003c7d RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4816.080855] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4816.080857] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4816.080859] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4816.080862] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4816.080865] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4816.080868] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4816.080870] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4816.080878] Call Trace: [ 4816.080883] [ 4816.080888] ? show_regs.cold.9+0x22/0x2f [ 4816.081424] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4816.081431] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4816.081436] ? watchdog+0x40/0x40 [ 4816.081440] ? hrtimer_interrupt+0x128/0x2c0 [ 4816.081443] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4816.081452] ? apic_timer_interrupt+0xf/0x20 [ 4816.081456] [ 4816.081459] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4816.081463] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4816.081468] do_raw_spin_lock+0x177/0x190 [ 4816.081473] _raw_spin_lock+0x21/0x30 [ 4816.081478] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4816.130401] [ 4816.130427] ? show_regs.cold.9+0x22/0x2f [ 4816.130441] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4816.130447] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4816.130451] ? watchdog+0x40/0x40 [ 4816.130456] ? hrtimer_interrupt+0x128/0x2c0 [ 4816.130460] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4816.130469] ? apic_timer_interrupt+0xf/0x20 [ 4816.130473] [ 4816.130476] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4816.130483] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4816.130487] do_raw_spin_lock+0x177/0x190 [ 4816.130492] _raw_spin_lock+0x21/0x30 [ 4816.130498] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4816.130771] obd_mod_free+0x46/0x140 [ptlrpc] [ 4816.131313] mdc_commit_open+0x91/0xb0 [mdc] [ 4816.131362] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4816.131579] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4816.132392] after_reply+0xe2c/0x1780 [ptlrpc] [ 4816.132696] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4816.133074] ? __next_timer_interrupt+0x160/0x160 [ 4816.133083] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4816.145040] process_one_work+0x2c8/0x700 [ 4816.145061] worker_thread+0x296/0x6e0 [ 4816.145066] ? rescuer_thread+0x570/0x570 [ 4816.145069] kthread+0x1d1/0x200 [ 4816.145073] ? set_kthread_struct+0x70/0x70 [ 4816.145076] ret_from_fork+0x1f/0x30 [ 4816.476313] ? do_wait_intr+0xf0/0xf0 [ 4816.477831] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4816.481027] kthread+0x1d1/0x200 [ 4816.482799] ? set_kthread_struct+0x70/0x70 [ 4816.485013] ret_from_fork+0x1f/0x30 [ 4840.059181] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4840.067988] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4840.117476] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4840.127723] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4840.137299] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4840.141370] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4840.155878] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4840.161938] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4840.167364] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4840.173972] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4840.180184] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4840.186286] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4840.192322] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4840.197780] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4840.209089] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4840.218863] Call Trace: [ 4840.223223] [ 4840.225391] ? show_regs.cold.9+0x22/0x2f [ 4840.230850] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4840.236968] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4840.241909] ? watchdog+0x40/0x40 [ 4840.245956] ? hrtimer_interrupt+0x128/0x2c0 [ 4840.249842] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4840.253319] ? apic_timer_interrupt+0xf/0x20 [ 4840.256940] [ 4840.258746] ? native_safe_halt+0x2e/0x50 [ 4840.261982] kvm_wait+0xcd/0xe0 [ 4840.264486] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4840.268786] do_raw_spin_lock+0x177/0x190 [ 4840.271903] _raw_spin_lock+0x21/0x30 [ 4840.275204] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4840.279772] ? do_raw_spin_unlock+0x75/0x190 [ 4840.283069] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4840.287833] ? do_wait_intr+0xf0/0xf0 [ 4840.290686] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4840.295676] mdc_reint+0x2d/0x110 [mdc] [ 4840.298057] mdc_unlink+0x658/0x870 [mdc] [ 4840.302303] lmv_unlink+0x34d/0xbd0 [lmv] [ 4840.305470] ll_unlink+0x24d/0x7f0 [lustre] [ 4840.308463] vfs_unlink+0x202/0x330 [ 4840.311291] do_unlinkat+0x2fa/0x460 [ 4840.314887] __x64_sys_unlink+0x37/0x50 [ 4840.317221] do_syscall_64+0xc1/0x440 [ 4840.319475] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4840.322820] RIP: 0033:0x7f33957fc10b [ 4840.325093] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4840.336393] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4840.341209] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4840.346172] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4840.350201] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4840.358049] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4840.365180] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4844.053028] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4844.058024] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4844.060973] Modules linked in: [ 4844.064001] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4844.064103] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4844.064113] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4844.064122] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4844.064515] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4844.064526] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4844.064530] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4844.064536] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4844.064538] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4844.064540] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4844.064542] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4844.064544] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4844.064548] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4844.064550] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4844.064552] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4844.064561] Call Trace: [ 4844.064566] [ 4844.064570] ? show_regs.cold.9+0x22/0x2f [ 4844.064579] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4844.064585] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4844.064588] ? watchdog+0x40/0x40 [ 4844.064592] ? hrtimer_interrupt+0x128/0x2c0 [ 4844.064595] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4844.064601] ? apic_timer_interrupt+0xf/0x20 [ 4844.064605] [ 4844.064607] ? native_safe_halt+0x2e/0x50 [ 4844.064610] kvm_wait+0xcd/0xe0 [ 4844.064616] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4844.064622] do_raw_spin_lock+0x177/0x190 [ 4844.064625] _raw_spin_lock+0x21/0x30 [ 4844.064628] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4844.064824] process_one_work+0x2c8/0x700 [ 4844.064830] worker_thread+0x296/0x6e0 [ 4844.064834] ? rescuer_thread+0x570/0x570 [ 4844.064836] kthread+0x1d1/0x200 [ 4844.064840] ? set_kthread_struct+0x70/0x70 [ 4844.064853] ret_from_fork+0x1f/0x30 [ 4844.339165] lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4844.376071] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4844.391256] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4844.399521] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4844.412222] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4844.448336] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 4844.462131] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4844.478602] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4844.483617] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4844.491016] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4844.494735] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4844.499571] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4844.504561] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4844.508313] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4844.511814] Call Trace: [ 4844.513000] [ 4844.514138] ? show_regs.cold.9+0x22/0x2f [ 4844.516049] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4844.518127] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4844.520363] ? watchdog+0x40/0x40 [ 4844.522122] ? hrtimer_interrupt+0x128/0x2c0 [ 4844.524152] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4844.526572] ? apic_timer_interrupt+0xf/0x20 [ 4844.528641] [ 4844.529775] ? native_safe_halt+0x2e/0x50 [ 4844.531761] kvm_wait+0xcd/0xe0 [ 4844.533377] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4844.539130] do_raw_spin_lock+0x177/0x190 [ 4844.541163] _raw_spin_lock+0x21/0x30 [ 4844.543699] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4844.547272] obd_mod_free+0x46/0x140 [ptlrpc] [ 4844.555119] mdc_commit_open+0x91/0xb0 [mdc] [ 4844.558313] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4844.564018] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4844.569051] after_reply+0xe2c/0x1780 [ptlrpc] [ 4844.572336] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4844.576985] ? __next_timer_interrupt+0x160/0x160 [ 4844.579376] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4844.582036] ? do_wait_intr+0xf0/0xf0 [ 4844.584082] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4844.587983] kthread+0x1d1/0x200 [ 4844.589569] ? set_kthread_struct+0x70/0x70 [ 4844.592115] ret_from_fork+0x1f/0x30 [ 4858.086604] rcu: INFO: rcu_sched self-detected stall on CPU [ 4858.093556] rcu: 0-....: (1719950 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=520275 [ 4858.101415] (t=2760066 jiffies g=297885 q=14183) [ 4858.102872] NMI backtrace for cpu 0 [ 4858.106109] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4858.118904] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4858.128904] Call Trace: [ 4858.133212] [ 4858.135589] ? dump_stack+0xbb/0x10e [ 4858.142768] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 4858.154120] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 4858.156198] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 4858.158859] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 4858.162384] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 4858.167583] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 4858.170773] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 4858.176981] ? update_process_times+0x6d/0xc0 [ 4858.182410] ? tick_sched_handle.isra.18+0x27/0x90 [ 4858.186207] ? tick_sched_timer+0x53/0xb0 [ 4858.188619] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4858.190771] ? tick_sched_handle.isra.18+0x90/0x90 [ 4858.193340] ? hrtimer_interrupt+0x128/0x2c0 [ 4858.195323] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4858.197413] ? apic_timer_interrupt+0xf/0x20 [ 4858.199632] [ 4858.201803] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4858.205038] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4858.211731] ? do_raw_spin_lock+0x177/0x190 [ 4858.213178] ? _raw_spin_lock+0x21/0x30 [ 4858.216566] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4858.222054] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 4858.225824] ? mdc_commit_open+0x91/0xb0 [mdc] [ 4858.232709] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4858.237974] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4858.246084] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 4858.250724] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4858.253583] ? __next_timer_interrupt+0x160/0x160 [ 4858.258652] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4858.264624] ? do_wait_intr+0xf0/0xf0 [ 4858.266064] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4858.271461] ? kthread+0x1d1/0x200 [ 4858.282078] ? set_kthread_struct+0x70/0x70 [ 4858.284807] ? ret_from_fork+0x1f/0x30 [ 4868.062130] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4868.073464] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4868.129462] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4868.137909] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4868.155391] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4868.160145] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4868.172401] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4868.179792] RAX: 0000000000007a08 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4868.185440] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4868.189154] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4868.193676] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4868.198735] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4868.203700] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4868.216001] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4868.218201] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4868.225663] Call Trace: [ 4868.226800] [ 4868.234572] ? show_regs.cold.9+0x22/0x2f [ 4868.237721] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4868.243408] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4868.246199] ? watchdog+0x40/0x40 [ 4868.247589] ? hrtimer_interrupt+0x128/0x2c0 [ 4868.250115] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4868.252168] ? apic_timer_interrupt+0xf/0x20 [ 4868.254410] [ 4868.255498] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4868.261319] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4868.267193] do_raw_spin_lock+0x177/0x190 [ 4868.271125] _raw_spin_lock+0x21/0x30 [ 4868.276089] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4868.286173] ? do_raw_spin_unlock+0x75/0x190 [ 4868.292490] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4868.295353] ? do_wait_intr+0xf0/0xf0 [ 4868.297669] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4868.305097] mdc_reint+0x2d/0x110 [mdc] [ 4868.310121] mdc_unlink+0x658/0x870 [mdc] [ 4868.312053] lmv_unlink+0x34d/0xbd0 [lmv] [ 4868.313734] ll_unlink+0x24d/0x7f0 [lustre] [ 4868.315678] vfs_unlink+0x202/0x330 [ 4868.317722] do_unlinkat+0x2fa/0x460 [ 4868.318999] __x64_sys_unlink+0x37/0x50 [ 4868.322114] do_syscall_64+0xc1/0x440 [ 4868.324363] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4868.327381] RIP: 0033:0x7f33957fc10b [ 4868.329274] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4868.340308] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4868.345993] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4868.348326] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4868.355352] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4868.359466] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4868.367545] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4872.058563] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4872.065496] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4872.104215] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4872.110214] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4872.116329] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4872.124751] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4872.128014] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4872.142186] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 4872.146486] RAX: 0000000000007f85 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4872.151134] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4872.153950] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4872.157361] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4872.160352] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4872.163171] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4872.167369] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4872.169508] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4872.172275] Call Trace: [ 4872.173288] [ 4872.174441] ? show_regs.cold.9+0x22/0x2f [ 4872.181445] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4872.182950] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4872.188463] ? watchdog+0x40/0x40 [ 4872.189566] ? hrtimer_interrupt+0x128/0x2c0 [ 4872.191477] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4872.195882] ? apic_timer_interrupt+0xf/0x20 [ 4872.199837] [ 4872.200953] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4872.203707] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4872.206311] do_raw_spin_lock+0x177/0x190 [ 4872.208086] _raw_spin_lock+0x21/0x30 [ 4872.210892] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4872.215645] process_one_work+0x2c8/0x700 [ 4872.217923] worker_thread+0x296/0x6e0 [ 4872.222799] ? rescuer_thread+0x570/0x570 [ 4872.225491] kthread+0x1d1/0x200 [ 4872.228141] ? set_kthread_struct+0x70/0x70 [ 4872.229731] ret_from_fork+0x1f/0x30 [ 4884.053146] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4884.061419] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4884.106483] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4884.117141] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4884.123307] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4884.125586] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4884.138505] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4884.147709] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4884.162752] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4884.168519] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4884.177466] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4884.187266] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4884.198915] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4884.210200] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4884.215141] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4884.221492] Call Trace: [ 4884.227616] [ 4884.228822] ? show_regs.cold.9+0x22/0x2f [ 4884.231202] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4884.234703] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4884.238328] ? watchdog+0x40/0x40 [ 4884.242286] ? hrtimer_interrupt+0x128/0x2c0 [ 4884.246264] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4884.257139] ? apic_timer_interrupt+0xf/0x20 [ 4884.265136] [ 4884.267204] ? native_safe_halt+0x2e/0x50 [ 4884.270164] kvm_wait+0xcd/0xe0 [ 4884.271493] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4884.276289] do_raw_spin_lock+0x177/0x190 [ 4884.278508] _raw_spin_lock+0x21/0x30 [ 4884.281179] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4884.291583] obd_mod_free+0x46/0x140 [ptlrpc] [ 4884.295500] mdc_commit_open+0x91/0xb0 [mdc] [ 4884.298097] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4884.302842] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4884.306236] after_reply+0xe2c/0x1780 [ptlrpc] [ 4884.312484] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4884.321718] ? __next_timer_interrupt+0x160/0x160 [ 4884.327278] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4884.329901] ? do_wait_intr+0xf0/0xf0 [ 4884.332134] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4884.335739] kthread+0x1d1/0x200 [ 4884.349435] ? set_kthread_struct+0x70/0x70 [ 4884.350916] ret_from_fork+0x1f/0x30 [ 4896.060127] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4896.064550] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4896.121058] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4896.133082] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4896.139907] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4896.144893] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4896.161313] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4896.170821] RAX: 0000000000003463 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4896.179204] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4896.184160] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4896.191210] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4896.199211] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4896.205610] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4896.220759] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4896.225958] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4896.232108] Call Trace: [ 4896.233613] [ 4896.235067] ? show_regs.cold.9+0x22/0x2f [ 4896.237738] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4896.240670] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4896.243004] ? watchdog+0x40/0x40 [ 4896.245924] ? hrtimer_interrupt+0x128/0x2c0 [ 4896.248311] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4896.254114] ? apic_timer_interrupt+0xf/0x20 [ 4896.261016] [ 4896.261938] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4896.267953] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4896.275240] do_raw_spin_lock+0x177/0x190 [ 4896.278910] _raw_spin_lock+0x21/0x30 [ 4896.281790] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4896.287690] ? do_raw_spin_unlock+0x75/0x190 [ 4896.290202] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4896.300159] ? do_wait_intr+0xf0/0xf0 [ 4896.301819] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4896.306944] mdc_reint+0x2d/0x110 [mdc] [ 4896.310003] mdc_unlink+0x658/0x870 [mdc] [ 4896.312809] lmv_unlink+0x34d/0xbd0 [lmv] [ 4896.316625] ll_unlink+0x24d/0x7f0 [lustre] [ 4896.321197] vfs_unlink+0x202/0x330 [ 4896.325247] do_unlinkat+0x2fa/0x460 [ 4896.327046] __x64_sys_unlink+0x37/0x50 [ 4896.328284] do_syscall_64+0xc1/0x440 [ 4896.331202] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4896.335889] RIP: 0033:0x7f33957fc10b [ 4896.338776] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4896.353811] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4896.361072] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4896.366058] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4896.371608] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4896.374867] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4896.378560] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4900.059762] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4900.069854] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4900.128754] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4900.139090] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4900.152335] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4900.162194] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4900.173561] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4900.198837] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 4900.205584] RAX: 0000000000005aa0 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4900.212074] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4900.219201] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4900.230505] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4900.236089] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4900.241709] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4900.247703] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4900.250046] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4900.255746] Call Trace: [ 4900.259605] [ 4900.260530] ? show_regs.cold.9+0x22/0x2f [ 4900.262146] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4900.267125] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4900.271813] ? watchdog+0x40/0x40 [ 4900.273187] ? hrtimer_interrupt+0x128/0x2c0 [ 4900.276294] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4900.277729] ? apic_timer_interrupt+0xf/0x20 [ 4900.279120] [ 4900.279883] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4900.281621] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4900.283392] do_raw_spin_lock+0x177/0x190 [ 4900.284751] _raw_spin_lock+0x21/0x30 [ 4900.286087] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4900.288200] process_one_work+0x2c8/0x700 [ 4900.289629] worker_thread+0x296/0x6e0 [ 4900.290798] ? rescuer_thread+0x570/0x570 [ 4900.292108] kthread+0x1d1/0x200 [ 4900.293212] ? set_kthread_struct+0x70/0x70 [ 4900.294650] ret_from_fork+0x1f/0x30 [ 4912.057133] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4912.065224] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4912.092442] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4912.100551] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4912.104467] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4912.107837] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4912.118339] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4912.122531] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4912.125857] RDX: 0000000000007bc1 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4912.129438] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4912.136132] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4912.139686] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4912.145730] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4912.155713] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4912.167724] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4912.177057] Call Trace: [ 4912.177963] [ 4912.188196] ? show_regs.cold.9+0x22/0x2f [ 4912.194977] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4912.202177] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4912.203647] ? watchdog+0x40/0x40 [ 4912.209267] ? hrtimer_interrupt+0x128/0x2c0 [ 4912.211731] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4912.219042] ? apic_timer_interrupt+0xf/0x20 [ 4912.223845] [ 4912.225565] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4912.231566] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4912.242376] do_raw_spin_lock+0x177/0x190 [ 4912.243894] _raw_spin_lock+0x21/0x30 [ 4912.250225] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4912.257251] obd_mod_free+0x46/0x140 [ptlrpc] [ 4912.259183] mdc_commit_open+0x91/0xb0 [mdc] [ 4912.264979] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4912.272829] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4912.278253] after_reply+0xe2c/0x1780 [ptlrpc] [ 4912.283004] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4912.291610] ? __next_timer_interrupt+0x160/0x160 [ 4912.298608] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4912.304584] ? do_wait_intr+0xf0/0xf0 [ 4912.305988] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4912.316602] kthread+0x1d1/0x200 [ 4912.323550] ? set_kthread_struct+0x70/0x70 [ 4912.331278] ret_from_fork+0x1f/0x30 [ 4924.059765] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4924.065206] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4924.117534] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4924.127557] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4924.136457] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4924.140491] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4924.157331] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4924.166653] RAX: 0000000000005b6a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4924.177473] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4924.183751] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4924.188314] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4924.193105] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4924.199901] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4924.205721] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4924.219303] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4924.228744] Call Trace: [ 4924.229996] [ 4924.231359] ? show_regs.cold.9+0x22/0x2f [ 4924.236414] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4924.239273] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4924.243612] ? watchdog+0x40/0x40 [ 4924.247031] ? hrtimer_interrupt+0x128/0x2c0 [ 4924.252453] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4924.256329] ? apic_timer_interrupt+0xf/0x20 [ 4924.258490] [ 4924.265917] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4924.273453] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4924.282978] do_raw_spin_lock+0x177/0x190 [ 4924.285405] _raw_spin_lock+0x21/0x30 [ 4924.289546] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4924.294201] ? do_raw_spin_unlock+0x75/0x190 [ 4924.297428] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4924.301212] ? do_wait_intr+0xf0/0xf0 [ 4924.303599] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4924.310069] mdc_reint+0x2d/0x110 [mdc] [ 4924.312357] mdc_unlink+0x658/0x870 [mdc] [ 4924.317003] lmv_unlink+0x34d/0xbd0 [lmv] [ 4924.320096] ll_unlink+0x24d/0x7f0 [lustre] [ 4924.322934] vfs_unlink+0x202/0x330 [ 4924.326286] do_unlinkat+0x2fa/0x460 [ 4924.330678] __x64_sys_unlink+0x37/0x50 [ 4924.333037] do_syscall_64+0xc1/0x440 [ 4924.335268] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4924.338443] RIP: 0033:0x7f33957fc10b [ 4924.342206] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4924.354709] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4924.359060] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4924.363400] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4924.368747] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4924.373564] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4924.378078] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4928.058582] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4928.070225] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4928.133481] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4928.148061] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4928.156324] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4928.159677] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4928.166272] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4928.179110] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4928.181473] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4928.188944] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4928.191809] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4928.196196] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4928.199511] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4928.203752] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4928.210043] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4928.214742] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4928.220473] Call Trace: [ 4928.221735] [ 4928.223505] ? show_regs.cold.9+0x22/0x2f [ 4928.227490] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4928.229454] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4928.231370] ? watchdog+0x40/0x40 [ 4928.232807] ? hrtimer_interrupt+0x128/0x2c0 [ 4928.240449] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4928.246698] ? apic_timer_interrupt+0xf/0x20 [ 4928.250986] [ 4928.253103] ? native_safe_halt+0x2e/0x50 [ 4928.258478] kvm_wait+0xcd/0xe0 [ 4928.261647] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4928.267540] do_raw_spin_lock+0x177/0x190 [ 4928.272301] _raw_spin_lock+0x21/0x30 [ 4928.275929] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4928.283591] process_one_work+0x2c8/0x700 [ 4928.285211] worker_thread+0x296/0x6e0 [ 4928.287613] ? rescuer_thread+0x570/0x570 [ 4928.291466] kthread+0x1d1/0x200 [ 4928.293591] ? set_kthread_struct+0x70/0x70 [ 4928.297571] ret_from_fork+0x1f/0x30 [ 4940.053053] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4940.066342] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4940.183876] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4940.214935] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4940.240996] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4940.251684] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4940.275825] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4940.287427] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4940.295679] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4940.305260] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4940.311642] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4940.315454] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4940.322756] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4940.329825] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4940.335966] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4940.342329] Call Trace: [ 4940.343878] [ 4940.346469] ? show_regs.cold.9+0x22/0x2f [ 4940.350201] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4940.355735] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4940.361561] ? watchdog+0x40/0x40 [ 4940.364214] ? hrtimer_interrupt+0x128/0x2c0 [ 4940.369390] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4940.373353] ? apic_timer_interrupt+0xf/0x20 [ 4940.377793] [ 4940.380465] ? native_safe_halt+0x2e/0x50 [ 4940.384542] kvm_wait+0xcd/0xe0 [ 4940.386208] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4940.391438] do_raw_spin_lock+0x177/0x190 [ 4940.398025] _raw_spin_lock+0x21/0x30 [ 4940.400130] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4940.405708] obd_mod_free+0x46/0x140 [ptlrpc] [ 4940.410345] mdc_commit_open+0x91/0xb0 [mdc] [ 4940.414443] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4940.418372] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4940.425650] after_reply+0xe2c/0x1780 [ptlrpc] [ 4940.432244] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4940.438903] ? __next_timer_interrupt+0x160/0x160 [ 4940.443984] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4940.448344] ? do_wait_intr+0xf0/0xf0 [ 4940.454426] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4940.457974] kthread+0x1d1/0x200 [ 4940.462348] ? set_kthread_struct+0x70/0x70 [ 4940.466938] ret_from_fork+0x1f/0x30 [ 4952.059270] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 4952.063968] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4952.100931] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4952.109709] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4952.113102] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4952.115843] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4952.127944] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4952.131301] RAX: 0000000000005287 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4952.134534] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4952.138467] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4952.144687] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4952.152225] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4952.157818] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4952.165291] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4952.172516] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4952.175465] Call Trace: [ 4952.176315] [ 4952.177053] ? show_regs.cold.9+0x22/0x2f [ 4952.182227] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4952.184666] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4952.186704] ? watchdog+0x40/0x40 [ 4952.188905] ? hrtimer_interrupt+0x128/0x2c0 [ 4952.193902] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4952.198539] ? apic_timer_interrupt+0xf/0x20 [ 4952.211207] [ 4952.213335] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4952.226330] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4952.236170] do_raw_spin_lock+0x177/0x190 [ 4952.238761] _raw_spin_lock+0x21/0x30 [ 4952.241614] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4952.248203] ? do_raw_spin_unlock+0x75/0x190 [ 4952.251574] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4952.254837] ? do_wait_intr+0xf0/0xf0 [ 4952.256453] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4952.260620] mdc_reint+0x2d/0x110 [mdc] [ 4952.262710] mdc_unlink+0x658/0x870 [mdc] [ 4952.267762] lmv_unlink+0x34d/0xbd0 [lmv] [ 4952.271851] ll_unlink+0x24d/0x7f0 [lustre] [ 4952.277308] vfs_unlink+0x202/0x330 [ 4952.279888] do_unlinkat+0x2fa/0x460 [ 4952.283804] __x64_sys_unlink+0x37/0x50 [ 4952.287819] do_syscall_64+0xc1/0x440 [ 4952.289355] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4952.294775] RIP: 0033:0x7f33957fc10b [ 4952.298482] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4952.319410] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4952.327303] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4952.333745] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4952.341528] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4952.344967] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4952.354220] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4956.058012] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4956.060851] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4956.079162] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4956.083340] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4956.086846] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4956.092911] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4956.098682] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 4956.114094] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4956.121014] RAX: 0000000000006246 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4956.124964] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4956.130257] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4956.135055] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4956.140996] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4956.147950] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4956.151094] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4956.153418] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4956.157335] Call Trace: [ 4956.158611] [ 4956.159790] ? show_regs.cold.9+0x22/0x2f [ 4956.161870] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4956.167583] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4956.171836] ? watchdog+0x40/0x40 [ 4956.173538] ? hrtimer_interrupt+0x128/0x2c0 [ 4956.175914] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4956.177742] ? apic_timer_interrupt+0xf/0x20 [ 4956.179750] [ 4956.180722] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 4956.182719] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4956.186329] do_raw_spin_lock+0x177/0x190 [ 4956.187792] _raw_spin_lock+0x21/0x30 [ 4956.189353] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4956.191661] process_one_work+0x2c8/0x700 [ 4956.193342] worker_thread+0x296/0x6e0 [ 4956.194863] ? rescuer_thread+0x570/0x570 [ 4956.196739] kthread+0x1d1/0x200 [ 4956.198005] ? set_kthread_struct+0x70/0x70 [ 4956.199828] ret_from_fork+0x1f/0x30 [ 4968.053539] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4968.060128] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4968.096604] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4968.102027] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4968.107851] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4968.111762] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 4968.123523] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 4968.127476] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4968.133984] RDX: 0000000000004d67 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4968.137848] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4968.141506] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4968.147193] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4968.150802] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4968.154401] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4968.156577] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4968.161821] Call Trace: [ 4968.164263] [ 4968.166309] ? show_regs.cold.9+0x22/0x2f [ 4968.169267] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4968.175729] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4968.178819] ? watchdog+0x40/0x40 [ 4968.183189] ? hrtimer_interrupt+0x128/0x2c0 [ 4968.185750] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4968.188383] ? apic_timer_interrupt+0xf/0x20 [ 4968.192888] [ 4968.193554] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 4968.195321] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4968.197748] do_raw_spin_lock+0x177/0x190 [ 4968.199442] _raw_spin_lock+0x21/0x30 [ 4968.203526] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4968.209681] obd_mod_free+0x46/0x140 [ptlrpc] [ 4968.211443] mdc_commit_open+0x91/0xb0 [mdc] [ 4968.214721] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4968.219188] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4968.225669] after_reply+0xe2c/0x1780 [ptlrpc] [ 4968.230521] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4968.233936] ? __next_timer_interrupt+0x160/0x160 [ 4968.238377] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4968.241158] ? do_wait_intr+0xf0/0xf0 [ 4968.244740] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4968.247105] kthread+0x1d1/0x200 [ 4968.248689] ? set_kthread_struct+0x70/0x70 [ 4968.250500] ret_from_fork+0x1f/0x30 [ 4980.059043] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 4980.062660] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4980.093634] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4980.104561] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4980.111043] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4980.113645] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4980.122153] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4980.126944] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4980.134043] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 4980.136508] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 4980.151077] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 4980.156477] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 4980.160450] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 4980.165961] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4980.170077] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 4980.177149] Call Trace: [ 4980.179608] [ 4980.180664] ? show_regs.cold.9+0x22/0x2f [ 4980.183945] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4980.189244] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4980.191658] ? watchdog+0x40/0x40 [ 4980.194368] ? hrtimer_interrupt+0x128/0x2c0 [ 4980.197498] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4980.200100] ? apic_timer_interrupt+0xf/0x20 [ 4980.203249] [ 4980.206140] ? native_safe_halt+0x2e/0x50 [ 4980.207447] kvm_wait+0xcd/0xe0 [ 4980.208814] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4980.211033] do_raw_spin_lock+0x177/0x190 [ 4980.212806] _raw_spin_lock+0x21/0x30 [ 4980.214244] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 4980.217016] ? do_raw_spin_unlock+0x75/0x190 [ 4980.218962] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 4980.221346] ? do_wait_intr+0xf0/0xf0 [ 4980.223054] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 4980.225429] mdc_reint+0x2d/0x110 [mdc] [ 4980.227280] mdc_unlink+0x658/0x870 [mdc] [ 4980.229178] lmv_unlink+0x34d/0xbd0 [lmv] [ 4980.230977] ll_unlink+0x24d/0x7f0 [lustre] [ 4980.233508] vfs_unlink+0x202/0x330 [ 4980.237209] do_unlinkat+0x2fa/0x460 [ 4980.239905] __x64_sys_unlink+0x37/0x50 [ 4980.241689] do_syscall_64+0xc1/0x440 [ 4980.245624] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 4980.249099] RIP: 0033:0x7f33957fc10b [ 4980.251447] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 4980.266963] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 4980.271992] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 4980.278673] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 4980.288737] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 4980.299993] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 4980.306379] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 4984.058068] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 4984.060919] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4984.082192] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4984.087219] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4984.091375] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 4984.099277] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4984.101157] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4984.112165] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4984.115969] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 4984.121220] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 4984.123691] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 4984.126768] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 4984.129665] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 4984.133054] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 4984.136588] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4984.139332] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 4984.142988] Call Trace: [ 4984.144486] [ 4984.146001] ? show_regs.cold.9+0x22/0x2f [ 4984.147913] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4984.150683] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4984.152733] ? watchdog+0x40/0x40 [ 4984.154852] ? hrtimer_interrupt+0x128/0x2c0 [ 4984.156331] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4984.158510] ? apic_timer_interrupt+0xf/0x20 [ 4984.161021] [ 4984.162177] ? native_safe_halt+0x2e/0x50 [ 4984.163998] kvm_wait+0xcd/0xe0 [ 4984.165336] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 4984.167739] do_raw_spin_lock+0x177/0x190 [ 4984.169860] _raw_spin_lock+0x21/0x30 [ 4984.171214] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 4984.173481] process_one_work+0x2c8/0x700 [ 4984.174998] worker_thread+0x296/0x6e0 [ 4984.176587] ? rescuer_thread+0x570/0x570 [ 4984.178426] kthread+0x1d1/0x200 [ 4984.179932] ? set_kthread_struct+0x70/0x70 [ 4984.181794] ret_from_fork+0x1f/0x30 [ 4996.053237] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 4996.057694] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 4996.082463] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 4996.088881] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4996.097619] RIP: 0010:native_safe_halt+0x2e/0x50 [ 4996.101860] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 4996.112580] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 4996.119897] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 4996.124741] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 4996.128941] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 4996.132308] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 4996.137206] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 4996.140921] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 4996.145123] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4996.148751] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 4996.152414] Call Trace: [ 4996.153483] [ 4996.154881] ? show_regs.cold.9+0x22/0x2f [ 4996.157118] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 4996.159448] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 4996.161753] ? watchdog+0x40/0x40 [ 4996.163600] ? hrtimer_interrupt+0x128/0x2c0 [ 4996.166098] ? smp_apic_timer_interrupt+0xa6/0x270 [ 4996.168921] ? apic_timer_interrupt+0xf/0x20 [ 4996.171125] [ 4996.171755] ? native_safe_halt+0x2e/0x50 [ 4996.173274] kvm_wait+0xcd/0xe0 [ 4996.174652] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 4996.177327] do_raw_spin_lock+0x177/0x190 [ 4996.179570] _raw_spin_lock+0x21/0x30 [ 4996.181484] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 4996.184531] obd_mod_free+0x46/0x140 [ptlrpc] [ 4996.188041] mdc_commit_open+0x91/0xb0 [mdc] [ 4996.189897] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 4996.192860] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 4996.195580] after_reply+0xe2c/0x1780 [ptlrpc] [ 4996.197933] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 4996.200606] ? __next_timer_interrupt+0x160/0x160 [ 4996.202898] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 4996.205766] ? do_wait_intr+0xf0/0xf0 [ 4996.207625] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 4996.210515] kthread+0x1d1/0x200 [ 4996.211611] ? set_kthread_struct+0x70/0x70 [ 4996.213852] ret_from_fork+0x1f/0x30 [ 5008.059820] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 5008.067993] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5008.095813] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5008.101220] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5008.105802] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5008.108984] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5008.118867] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5008.122834] RAX: 0000000000007a24 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5008.126535] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5008.130307] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5008.133977] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5008.137750] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5008.141109] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5008.144868] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5008.147907] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5008.151689] Call Trace: [ 5008.153243] [ 5008.154205] ? show_regs.cold.9+0x22/0x2f [ 5008.156803] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5008.159165] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5008.161032] ? watchdog+0x40/0x40 [ 5008.162841] ? hrtimer_interrupt+0x128/0x2c0 [ 5008.165578] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5008.168019] ? apic_timer_interrupt+0xf/0x20 [ 5008.170046] [ 5008.171074] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5008.173815] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5008.176782] do_raw_spin_lock+0x177/0x190 [ 5008.178781] _raw_spin_lock+0x21/0x30 [ 5008.180807] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5008.183697] ? do_raw_spin_unlock+0x75/0x190 [ 5008.187105] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5008.191289] ? do_wait_intr+0xf0/0xf0 [ 5008.194608] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5008.197623] mdc_reint+0x2d/0x110 [mdc] [ 5008.199624] mdc_unlink+0x658/0x870 [mdc] [ 5008.201680] lmv_unlink+0x34d/0xbd0 [lmv] [ 5008.203843] ll_unlink+0x24d/0x7f0 [lustre] [ 5008.206847] vfs_unlink+0x202/0x330 [ 5008.208777] do_unlinkat+0x2fa/0x460 [ 5008.210572] __x64_sys_unlink+0x37/0x50 [ 5008.212273] do_syscall_64+0xc1/0x440 [ 5008.214613] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5008.216940] RIP: 0033:0x7f33957fc10b [ 5008.218716] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5008.227264] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5008.230676] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5008.233779] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5008.236884] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5008.240559] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5008.243699] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5012.058047] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5012.061203] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5012.082956] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5012.087739] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5012.091158] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5012.093615] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5012.095270] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5012.102148] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5012.105195] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5012.108144] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5012.111230] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5012.113501] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5012.116287] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5012.119129] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5012.122407] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5012.124791] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5012.128016] Call Trace: [ 5012.128905] [ 5012.129881] ? show_regs.cold.9+0x22/0x2f [ 5012.131430] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5012.133436] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5012.135248] ? watchdog+0x40/0x40 [ 5012.136710] ? hrtimer_interrupt+0x128/0x2c0 [ 5012.138577] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5012.140528] ? apic_timer_interrupt+0xf/0x20 [ 5012.142281] [ 5012.143333] ? native_safe_halt+0x2e/0x50 [ 5012.145046] kvm_wait+0xcd/0xe0 [ 5012.146294] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5012.148545] do_raw_spin_lock+0x177/0x190 [ 5012.150244] _raw_spin_lock+0x21/0x30 [ 5012.151960] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5012.154668] process_one_work+0x2c8/0x700 [ 5012.156407] worker_thread+0x296/0x6e0 [ 5012.157874] ? rescuer_thread+0x570/0x570 [ 5012.159674] kthread+0x1d1/0x200 [ 5012.161075] ? set_kthread_struct+0x70/0x70 [ 5012.162751] ret_from_fork+0x1f/0x30 [ 5024.053020] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 5024.057773] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5024.079929] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5024.084765] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5024.089294] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5024.092757] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5024.112705] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5024.120726] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5024.127024] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5024.134753] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5024.146561] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5024.162013] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5024.166107] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5024.177353] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5024.180593] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5024.187336] Call Trace: [ 5024.188671] [ 5024.189875] ? show_regs.cold.9+0x22/0x2f [ 5024.191277] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5024.195264] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5024.197496] ? watchdog+0x40/0x40 [ 5024.202038] ? hrtimer_interrupt+0x128/0x2c0 [ 5024.206079] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5024.207749] ? apic_timer_interrupt+0xf/0x20 [ 5024.215591] [ 5024.216627] ? native_safe_halt+0x2e/0x50 [ 5024.227971] kvm_wait+0xcd/0xe0 [ 5024.229810] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5024.237672] do_raw_spin_lock+0x177/0x190 [ 5024.241759] _raw_spin_lock+0x21/0x30 [ 5024.245021] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5024.249080] obd_mod_free+0x46/0x140 [ptlrpc] [ 5024.252981] mdc_commit_open+0x91/0xb0 [mdc] [ 5024.254495] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5024.259937] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5024.265909] after_reply+0xe2c/0x1780 [ptlrpc] [ 5024.271216] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5024.273273] ? __next_timer_interrupt+0x160/0x160 [ 5024.275544] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5024.277876] ? do_wait_intr+0xf0/0xf0 [ 5024.279637] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5024.285477] kthread+0x1d1/0x200 [ 5024.286716] ? set_kthread_struct+0x70/0x70 [ 5024.292144] ret_from_fork+0x1f/0x30 [ 5036.059595] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 5036.063591] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5036.085333] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5036.089444] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5036.093276] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5036.096544] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5036.105334] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5036.109107] RAX: 000000000000138f RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5036.112220] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5036.117372] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5036.121149] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5036.124097] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5036.127204] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5036.132861] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5036.137456] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5036.142140] Call Trace: [ 5036.143506] [ 5036.144468] ? show_regs.cold.9+0x22/0x2f [ 5036.147600] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5036.150172] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5036.152818] ? watchdog+0x40/0x40 [ 5036.154292] ? hrtimer_interrupt+0x128/0x2c0 [ 5036.160417] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5036.163044] ? apic_timer_interrupt+0xf/0x20 [ 5036.165178] [ 5036.166119] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5036.168038] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5036.170408] do_raw_spin_lock+0x177/0x190 [ 5036.174536] _raw_spin_lock+0x21/0x30 [ 5036.178425] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5036.183909] ? do_raw_spin_unlock+0x75/0x190 [ 5036.188669] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5036.195266] ? do_wait_intr+0xf0/0xf0 [ 5036.197185] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5036.200042] mdc_reint+0x2d/0x110 [mdc] [ 5036.201631] mdc_unlink+0x658/0x870 [mdc] [ 5036.205379] lmv_unlink+0x34d/0xbd0 [lmv] [ 5036.207644] ll_unlink+0x24d/0x7f0 [lustre] [ 5036.211161] vfs_unlink+0x202/0x330 [ 5036.212775] do_unlinkat+0x2fa/0x460 [ 5036.214069] __x64_sys_unlink+0x37/0x50 [ 5036.215995] do_syscall_64+0xc1/0x440 [ 5036.217862] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5036.221292] RIP: 0033:0x7f33957fc10b [ 5036.223055] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5036.233281] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5036.238009] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5036.243491] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5036.247807] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5036.250473] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5036.256344] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5038.090786] rcu: INFO: rcu_sched self-detected stall on CPU [ 5038.097329] rcu: 0-....: (1827490 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=554398 [ 5038.106306] (t=2940070 jiffies g=297885 q=14937) [ 5038.113138] NMI backtrace for cpu 0 [ 5038.115904] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5038.127607] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5038.135277] Call Trace: [ 5038.137188] [ 5038.138094] ? dump_stack+0xbb/0x10e [ 5038.140708] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 5038.144260] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 5038.148940] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 5038.151399] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 5038.157148] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 5038.162941] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 5038.166895] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 5038.169638] ? update_process_times+0x6d/0xc0 [ 5038.171165] ? tick_sched_handle.isra.18+0x27/0x90 [ 5038.173349] ? tick_sched_timer+0x53/0xb0 [ 5038.175428] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5038.178398] ? tick_sched_handle.isra.18+0x90/0x90 [ 5038.182939] ? hrtimer_interrupt+0x128/0x2c0 [ 5038.184880] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5038.188046] ? apic_timer_interrupt+0xf/0x20 [ 5038.190357] [ 5038.193732] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5038.198589] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5038.203988] ? do_raw_spin_lock+0x177/0x190 [ 5038.206084] ? _raw_spin_lock+0x21/0x30 [ 5038.211788] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5038.216847] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 5038.221372] ? mdc_commit_open+0x91/0xb0 [mdc] [ 5038.224686] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5038.228917] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5038.233310] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 5038.239853] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5038.246200] ? __next_timer_interrupt+0x160/0x160 [ 5038.249761] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5038.255134] ? do_wait_intr+0xf0/0xf0 [ 5038.263706] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5038.276895] ? kthread+0x1d1/0x200 [ 5038.281228] ? set_kthread_struct+0x70/0x70 [ 5038.289136] ? ret_from_fork+0x1f/0x30 [ 5040.058189] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5040.066798] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5040.101145] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5040.106295] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5040.110855] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5040.115006] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5040.117851] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5040.127432] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5040.141324] RAX: 0000000000000588 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5040.148055] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5040.155205] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5040.161730] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5040.172047] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5040.180061] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5040.185114] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5040.187946] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5040.192415] Call Trace: [ 5040.195869] [ 5040.197774] ? show_regs.cold.9+0x22/0x2f [ 5040.201176] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5040.207048] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5040.211755] ? watchdog+0x40/0x40 [ 5040.213735] ? hrtimer_interrupt+0x128/0x2c0 [ 5040.217015] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5040.220378] ? apic_timer_interrupt+0xf/0x20 [ 5040.222864] [ 5040.224310] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5040.227199] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5040.230090] do_raw_spin_lock+0x177/0x190 [ 5040.235268] _raw_spin_lock+0x21/0x30 [ 5040.239697] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5040.243041] process_one_work+0x2c8/0x700 [ 5040.245725] worker_thread+0x296/0x6e0 [ 5040.247889] ? rescuer_thread+0x570/0x570 [ 5040.250751] kthread+0x1d1/0x200 [ 5040.253472] ? set_kthread_struct+0x70/0x70 [ 5040.255804] ret_from_fork+0x1f/0x30 [ 5064.053087] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 5064.056969] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5064.059027] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 5064.093479] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5064.098614] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5064.098701] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5064.098711] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5064.098715] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5064.098730] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5064.098735] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 5064.098740] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5064.098742] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5064.098744] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5064.098746] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5064.098749] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5064.098753] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5064.098755] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5064.098758] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5064.098768] Call Trace: [ 5064.098775] [ 5064.098782] ? show_regs.cold.9+0x22/0x2f [ 5064.098795] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5064.098802] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5064.098807] ? watchdog+0x40/0x40 [ 5064.098811] ? hrtimer_interrupt+0x128/0x2c0 [ 5064.098815] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5064.098823] ? apic_timer_interrupt+0xf/0x20 [ 5064.098827] [ 5064.098831] ? native_safe_halt+0x2e/0x50 [ 5064.098835] kvm_wait+0xcd/0xe0 [ 5064.098842] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5064.098850] do_raw_spin_lock+0x177/0x190 [ 5064.098854] _raw_spin_lock+0x21/0x30 [ 5064.098859] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5064.106463] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5064.106474] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5064.106490] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5064.106494] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5064.106501] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5064.133721] ? do_raw_spin_unlock+0x75/0x190 [ 5064.139135] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5064.139147] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5064.139150] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5064.139153] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5064.139156] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5064.139159] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5064.139161] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5064.139170] Call Trace: [ 5064.139178] [ 5064.139186] ? show_regs.cold.9+0x22/0x2f [ 5064.139203] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5064.139210] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5064.139214] ? watchdog+0x40/0x40 [ 5064.139219] ? hrtimer_interrupt+0x128/0x2c0 [ 5064.139222] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5064.139231] ? apic_timer_interrupt+0xf/0x20 [ 5064.139235] [ 5064.139237] ? native_safe_halt+0x2e/0x50 [ 5064.139242] kvm_wait+0xcd/0xe0 [ 5064.139251] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5064.139259] do_raw_spin_lock+0x177/0x190 [ 5064.139263] _raw_spin_lock+0x21/0x30 [ 5064.139269] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5064.144486] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5064.146470] obd_mod_free+0x46/0x140 [ptlrpc] [ 5064.158319] ? do_wait_intr+0xf0/0xf0 [ 5064.158344] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5064.159191] mdc_reint+0x2d/0x110 [mdc] [ 5064.159232] mdc_unlink+0x658/0x870 [mdc] [ 5064.159262] lmv_unlink+0x34d/0xbd0 [lmv] [ 5064.159300] ll_unlink+0x24d/0x7f0 [lustre] [ 5064.159443] vfs_unlink+0x202/0x330 [ 5064.159454] do_unlinkat+0x2fa/0x460 [ 5064.159460] __x64_sys_unlink+0x37/0x50 [ 5064.159464] do_syscall_64+0xc1/0x440 [ 5064.159473] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5064.159485] RIP: 0033:0x7f33957fc10b [ 5064.159493] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5064.159498] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5064.159504] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5064.159506] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5064.159509] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5064.159511] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5064.159514] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5064.540816] mdc_commit_open+0x91/0xb0 [mdc] [ 5064.543930] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5064.548228] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5064.553215] after_reply+0xe2c/0x1780 [ptlrpc] [ 5064.556580] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5064.561572] ? __next_timer_interrupt+0x160/0x160 [ 5064.565863] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5064.570144] ? do_wait_intr+0xf0/0xf0 [ 5064.574874] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5064.580477] kthread+0x1d1/0x200 [ 5064.585024] ? set_kthread_struct+0x70/0x70 [ 5064.586984] ret_from_fork+0x1f/0x30 [ 5068.061625] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 5068.072319] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5068.115855] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5068.121850] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5068.130386] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5068.138523] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5068.149730] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5068.165644] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5068.185746] RAX: 0000000000005e86 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5068.193647] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5068.196999] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5068.199366] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5068.202444] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5068.210456] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5068.225809] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5068.228277] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5068.233841] Call Trace: [ 5068.234895] [ 5068.236671] ? show_regs.cold.9+0x22/0x2f [ 5068.240236] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5068.243068] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5068.250114] ? watchdog+0x40/0x40 [ 5068.253212] ? hrtimer_interrupt+0x128/0x2c0 [ 5068.254922] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5068.256747] ? apic_timer_interrupt+0xf/0x20 [ 5068.260662] [ 5068.267574] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5068.270018] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5068.273417] do_raw_spin_lock+0x177/0x190 [ 5068.278302] _raw_spin_lock+0x21/0x30 [ 5068.281361] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5068.285478] process_one_work+0x2c8/0x700 [ 5068.287261] worker_thread+0x296/0x6e0 [ 5068.288619] ? rescuer_thread+0x570/0x570 [ 5068.292328] kthread+0x1d1/0x200 [ 5068.296138] ? set_kthread_struct+0x70/0x70 [ 5068.299530] ret_from_fork+0x1f/0x30 [ 5092.058529] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 5092.059628] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 5092.073971] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5092.074088] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5092.074097] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5092.074101] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5092.074117] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5092.074122] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5092.074127] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5092.074129] RDX: 00000000000046a6 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5092.074132] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5092.074134] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5092.074136] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5092.074140] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5092.074143] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5092.074145] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5092.074155] Call Trace: [ 5092.074161] [ 5092.074167] ? show_regs.cold.9+0x22/0x2f [ 5092.074175] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5092.074182] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5092.074186] ? watchdog+0x40/0x40 [ 5092.074190] ? hrtimer_interrupt+0x128/0x2c0 [ 5092.074194] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5092.074203] ? apic_timer_interrupt+0xf/0x20 [ 5092.074208] [ 5092.074210] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5092.074215] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5092.074219] do_raw_spin_lock+0x177/0x190 [ 5092.074224] _raw_spin_lock+0x21/0x30 [ 5092.074229] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5092.074506] obd_mod_free+0x46/0x140 [ptlrpc] [ 5092.074697] mdc_commit_open+0x91/0xb0 [mdc] [ 5092.074742] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5092.074944] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5092.075136] after_reply+0xe2c/0x1780 [ptlrpc] [ 5092.075328] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5092.075517] ? __next_timer_interrupt+0x160/0x160 [ 5092.075526] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5092.075724] ? do_wait_intr+0xf0/0xf0 [ 5092.075733] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5092.075945] kthread+0x1d1/0x200 [ 5092.075952] ? set_kthread_struct+0x70/0x70 [ 5092.075956] ret_from_fork+0x1f/0x30 [ 5092.347912] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5092.387964] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5092.401421] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5092.406334] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5092.411387] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5092.430368] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5092.436418] RAX: 0000000000007a89 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5092.442941] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5092.454716] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5092.459230] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5092.462962] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5092.466230] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5092.469707] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5092.471421] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5092.475988] Call Trace: [ 5092.478235] [ 5092.479546] ? show_regs.cold.9+0x22/0x2f [ 5092.481218] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5092.485638] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5092.489881] ? watchdog+0x40/0x40 [ 5092.491833] ? hrtimer_interrupt+0x128/0x2c0 [ 5092.497130] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5092.500627] ? apic_timer_interrupt+0xf/0x20 [ 5092.503738] [ 5092.505277] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5092.507137] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5092.510232] do_raw_spin_lock+0x177/0x190 [ 5092.512054] _raw_spin_lock+0x21/0x30 [ 5092.513984] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5092.519171] ? do_raw_spin_unlock+0x75/0x190 [ 5092.521245] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5092.525658] ? do_wait_intr+0xf0/0xf0 [ 5092.527487] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5092.532074] mdc_reint+0x2d/0x110 [mdc] [ 5092.534158] mdc_unlink+0x658/0x870 [mdc] [ 5092.538793] lmv_unlink+0x34d/0xbd0 [lmv] [ 5092.541258] ll_unlink+0x24d/0x7f0 [lustre] [ 5092.545327] vfs_unlink+0x202/0x330 [ 5092.548464] do_unlinkat+0x2fa/0x460 [ 5092.550650] __x64_sys_unlink+0x37/0x50 [ 5092.552681] do_syscall_64+0xc1/0x440 [ 5092.554939] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5092.559901] RIP: 0033:0x7f33957fc10b [ 5092.562952] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5092.587897] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5092.591269] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5092.594829] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5092.598785] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5092.602828] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5092.607838] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5096.059612] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 5096.065238] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5096.120048] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5096.133584] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5096.146239] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5096.152535] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5096.156756] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5096.168620] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5096.171772] RAX: 0000000000007526 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5096.175954] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5096.178566] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5096.181207] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5096.184857] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5096.187864] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5096.194435] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5096.196678] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5096.199070] Call Trace: [ 5096.201232] [ 5096.202123] ? show_regs.cold.9+0x22/0x2f [ 5096.206023] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5096.210249] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5096.213799] ? watchdog+0x40/0x40 [ 5096.215475] ? hrtimer_interrupt+0x128/0x2c0 [ 5096.220301] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5096.223001] ? apic_timer_interrupt+0xf/0x20 [ 5096.230651] [ 5096.231732] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5096.235688] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5096.239151] do_raw_spin_lock+0x177/0x190 [ 5096.240762] _raw_spin_lock+0x21/0x30 [ 5096.242641] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5096.244993] process_one_work+0x2c8/0x700 [ 5096.247829] worker_thread+0x296/0x6e0 [ 5096.249535] ? rescuer_thread+0x570/0x570 [ 5096.256196] kthread+0x1d1/0x200 [ 5096.257285] ? set_kthread_struct+0x70/0x70 [ 5096.258505] ret_from_fork+0x1f/0x30 [ 5120.057776] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5120.059113] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5120.067954] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5120.068047] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5120.073997] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5120.074087] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5120.074094] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5120.074097] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5120.074161] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5120.074166] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5120.074171] RAX: 000000000000552e RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5120.074173] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5120.074175] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5120.074177] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5120.074179] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5120.074183] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5120.074185] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5120.074187] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5120.074195] Call Trace: [ 5120.075595] [ 5120.114696] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5120.114712] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5120.114737] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5120.114742] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5120.114750] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5120.114753] RDX: 0000000000007b78 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5120.114756] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5120.115366] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5120.115370] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5120.115435] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5120.115439] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5120.115443] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5120.115807] Call Trace: [ 5120.115815] [ 5120.115823] ? show_regs.cold.9+0x22/0x2f [ 5120.115835] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5120.115841] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5120.115847] ? watchdog+0x40/0x40 [ 5120.115851] ? hrtimer_interrupt+0x128/0x2c0 [ 5120.115856] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5120.115866] ? apic_timer_interrupt+0xf/0x20 [ 5120.115884] [ 5120.115888] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5120.115894] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5120.115899] do_raw_spin_lock+0x177/0x190 [ 5120.115903] _raw_spin_lock+0x21/0x30 [ 5120.115910] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5120.124698] ? show_regs.cold.9+0x22/0x2f [ 5120.171199] obd_mod_free+0x46/0x140 [ptlrpc] [ 5120.181058] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5120.181080] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5120.181085] ? watchdog+0x40/0x40 [ 5120.181090] ? hrtimer_interrupt+0x128/0x2c0 [ 5120.181094] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5120.181103] ? apic_timer_interrupt+0xf/0x20 [ 5120.181108] [ 5120.181113] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5120.181120] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5120.181124] do_raw_spin_lock+0x177/0x190 [ 5120.181130] _raw_spin_lock+0x21/0x30 [ 5120.181136] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5120.181430] ? do_raw_spin_unlock+0x75/0x190 [ 5120.181436] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5120.181739] ? do_wait_intr+0xf0/0xf0 [ 5120.181747] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5120.182146] mdc_reint+0x2d/0x110 [mdc] [ 5120.182184] mdc_unlink+0x658/0x870 [mdc] [ 5120.182214] lmv_unlink+0x34d/0xbd0 [lmv] [ 5120.182250] ll_unlink+0x24d/0x7f0 [lustre] [ 5120.182393] vfs_unlink+0x202/0x330 [ 5120.182402] do_unlinkat+0x2fa/0x460 [ 5120.182407] __x64_sys_unlink+0x37/0x50 [ 5120.182410] do_syscall_64+0xc1/0x440 [ 5120.182418] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5120.182426] RIP: 0033:0x7f33957fc10b [ 5120.182433] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5120.182438] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5120.182443] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5120.182445] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5120.182448] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5120.182450] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5120.182452] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5120.545808] mdc_commit_open+0x91/0xb0 [mdc] [ 5120.551256] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5120.557286] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5120.561765] after_reply+0xe2c/0x1780 [ptlrpc] [ 5120.568619] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5120.574958] ? __next_timer_interrupt+0x160/0x160 [ 5120.579144] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5120.583411] ? do_wait_intr+0xf0/0xf0 [ 5120.584622] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5120.594562] kthread+0x1d1/0x200 [ 5120.598569] ? set_kthread_struct+0x70/0x70 [ 5120.602927] ret_from_fork+0x1f/0x30 [ 5124.058295] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 5124.062134] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5124.087775] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5124.097102] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5124.104438] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5124.108335] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5124.111387] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5124.123008] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5124.130893] RAX: 0000000000002206 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5124.133067] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5124.136187] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5124.139642] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5124.146572] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5124.149623] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5124.153687] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5124.156382] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5124.161559] Call Trace: [ 5124.164903] [ 5124.165668] ? show_regs.cold.9+0x22/0x2f [ 5124.168573] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5124.171972] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5124.175708] ? watchdog+0x40/0x40 [ 5124.177282] ? hrtimer_interrupt+0x128/0x2c0 [ 5124.182336] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5124.187221] ? apic_timer_interrupt+0xf/0x20 [ 5124.189144] [ 5124.191497] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5124.198659] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5124.205563] do_raw_spin_lock+0x177/0x190 [ 5124.207973] _raw_spin_lock+0x21/0x30 [ 5124.214392] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5124.218437] process_one_work+0x2c8/0x700 [ 5124.220430] worker_thread+0x296/0x6e0 [ 5124.223770] ? rescuer_thread+0x570/0x570 [ 5124.225162] kthread+0x1d1/0x200 [ 5124.227304] ? set_kthread_struct+0x70/0x70 [ 5124.229624] ret_from_fork+0x1f/0x30 [ 5148.053904] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5148.060186] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5148.061859] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5148.061941] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5148.061948] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5148.061950] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5148.061968] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5148.061972] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5148.061977] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5148.061979] RDX: 0000000000002d13 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5148.061981] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5148.061983] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5148.061985] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5148.061989] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5148.061991] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5148.061994] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5148.062003] Call Trace: [ 5148.062010] [ 5148.062017] ? show_regs.cold.9+0x22/0x2f [ 5148.062027] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5148.062033] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5148.062038] ? watchdog+0x40/0x40 [ 5148.062041] ? hrtimer_interrupt+0x128/0x2c0 [ 5148.062045] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5148.062054] ? apic_timer_interrupt+0xf/0x20 [ 5148.062059] [ 5148.062061] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5148.062066] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5148.062071] do_raw_spin_lock+0x177/0x190 [ 5148.062075] _raw_spin_lock+0x21/0x30 [ 5148.062081] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5148.062613] obd_mod_free+0x46/0x140 [ptlrpc] [ 5148.062819] mdc_commit_open+0x91/0xb0 [mdc] [ 5148.062860] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5148.063050] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5148.063239] after_reply+0xe2c/0x1780 [ptlrpc] [ 5148.063600] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5148.064960] ? __next_timer_interrupt+0x160/0x160 [ 5148.064975] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5148.065241] ? do_wait_intr+0xf0/0xf0 [ 5148.065252] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5148.065522] kthread+0x1d1/0x200 [ 5148.065528] ? set_kthread_struct+0x70/0x70 [ 5148.065532] ret_from_fork+0x1f/0x30 [ 5148.361381] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5148.404990] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5148.430026] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5148.441591] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5148.450805] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5148.469292] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5148.471419] RAX: 0000000000005f26 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5148.485540] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5148.488750] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5148.493570] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5148.497260] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5148.502801] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5148.511570] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5148.518684] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5148.522495] Call Trace: [ 5148.528495] [ 5148.529287] ? show_regs.cold.9+0x22/0x2f [ 5148.530720] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5148.534531] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5148.537694] ? watchdog+0x40/0x40 [ 5148.543769] ? hrtimer_interrupt+0x128/0x2c0 [ 5148.545046] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5148.552855] ? apic_timer_interrupt+0xf/0x20 [ 5148.555246] [ 5148.556491] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5148.559652] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5148.567928] do_raw_spin_lock+0x177/0x190 [ 5148.576697] _raw_spin_lock+0x21/0x30 [ 5148.578046] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5148.583003] ? do_raw_spin_unlock+0x75/0x190 [ 5148.592217] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5148.605750] ? do_wait_intr+0xf0/0xf0 [ 5148.606865] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5148.620848] mdc_reint+0x2d/0x110 [mdc] [ 5148.623157] mdc_unlink+0x658/0x870 [mdc] [ 5148.625461] lmv_unlink+0x34d/0xbd0 [lmv] [ 5148.627151] ll_unlink+0x24d/0x7f0 [lustre] [ 5148.632077] vfs_unlink+0x202/0x330 [ 5148.634203] do_unlinkat+0x2fa/0x460 [ 5148.637597] __x64_sys_unlink+0x37/0x50 [ 5148.644758] do_syscall_64+0xc1/0x440 [ 5148.647690] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5148.655752] RIP: 0033:0x7f33957fc10b [ 5148.657703] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5148.669362] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5148.672836] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5148.676728] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5148.680262] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5148.684791] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5148.689760] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5152.058012] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 5152.061328] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5152.117422] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5152.137040] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5152.151543] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5152.162172] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5152.171222] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5152.185190] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5152.196920] RAX: 00000000000013c0 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5152.203620] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5152.209879] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5152.219212] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5152.230197] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5152.236506] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5152.241887] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5152.249013] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5152.257582] Call Trace: [ 5152.260353] [ 5152.262072] ? show_regs.cold.9+0x22/0x2f [ 5152.266004] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5152.269588] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5152.277010] ? watchdog+0x40/0x40 [ 5152.279453] ? hrtimer_interrupt+0x128/0x2c0 [ 5152.282190] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5152.285536] ? apic_timer_interrupt+0xf/0x20 [ 5152.288586] [ 5152.290478] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5152.293448] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5152.296494] do_raw_spin_lock+0x177/0x190 [ 5152.298776] _raw_spin_lock+0x21/0x30 [ 5152.301069] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5152.304244] process_one_work+0x2c8/0x700 [ 5152.306242] worker_thread+0x296/0x6e0 [ 5152.308215] ? rescuer_thread+0x570/0x570 [ 5152.312675] kthread+0x1d1/0x200 [ 5152.314139] ? set_kthread_struct+0x70/0x70 [ 5152.320199] ret_from_fork+0x1f/0x30 [ 5176.053772] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5176.061695] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5176.062538] Modules linked in: lustre(O) [ 5176.068121] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5176.068208] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5176.068216] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5176.068219] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5176.068233] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5176.068238] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5176.068243] RAX: 0000000000005f19 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5176.068245] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5176.068247] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5176.068249] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5176.068251] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5176.068255] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5176.068258] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5176.068260] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5176.068269] Call Trace: [ 5176.068273] [ 5176.068280] ? show_regs.cold.9+0x22/0x2f [ 5176.068289] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5176.068296] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5176.068301] ? watchdog+0x40/0x40 [ 5176.068305] ? hrtimer_interrupt+0x128/0x2c0 [ 5176.068309] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5176.068319] ? apic_timer_interrupt+0xf/0x20 [ 5176.068324] [ 5176.068327] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5176.068333] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5176.068414] do_raw_spin_lock+0x177/0x190 [ 5176.068419] _raw_spin_lock+0x21/0x30 [ 5176.068424] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5176.068975] ? do_raw_spin_unlock+0x75/0x190 [ 5176.068982] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5176.130698] osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5176.130800] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5176.130810] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5176.130814] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5176.130832] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5176.130836] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5176.130841] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5176.130844] RDX: 000000000000679e RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5176.130846] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5176.130848] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5176.130851] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5176.130854] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5176.130857] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5176.130859] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5176.130866] Call Trace: [ 5176.130874] [ 5176.131195] ? show_regs.cold.9+0x22/0x2f [ 5176.131208] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5176.131214] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5176.131219] ? watchdog+0x40/0x40 [ 5176.131224] ? hrtimer_interrupt+0x128/0x2c0 [ 5176.131227] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5176.131237] ? apic_timer_interrupt+0xf/0x20 [ 5176.131241] [ 5176.131245] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5176.131249] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5176.131254] do_raw_spin_lock+0x177/0x190 [ 5176.131259] _raw_spin_lock+0x21/0x30 [ 5176.131264] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5176.132146] obd_mod_free+0x46/0x140 [ptlrpc] [ 5176.132439] mdc_commit_open+0x91/0xb0 [mdc] [ 5176.132549] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5176.133004] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5176.133222] after_reply+0xe2c/0x1780 [ptlrpc] [ 5176.140138] ? do_wait_intr+0xf0/0xf0 [ 5176.140161] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5176.140885] mdc_reint+0x2d/0x110 [mdc] [ 5176.140925] mdc_unlink+0x658/0x870 [mdc] [ 5176.140951] lmv_unlink+0x34d/0xbd0 [lmv] [ 5176.140985] ll_unlink+0x24d/0x7f0 [lustre] [ 5176.141329] vfs_unlink+0x202/0x330 [ 5176.141402] do_unlinkat+0x2fa/0x460 [ 5176.141407] __x64_sys_unlink+0x37/0x50 [ 5176.141411] do_syscall_64+0xc1/0x440 [ 5176.141417] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5176.141425] RIP: 0033:0x7f33957fc10b [ 5176.141432] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5176.141437] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5176.141442] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5176.141444] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5176.141446] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5176.141448] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5176.141451] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5176.451307] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5176.456512] ? __next_timer_interrupt+0x160/0x160 [ 5176.459187] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5176.462959] ? do_wait_intr+0xf0/0xf0 [ 5176.465639] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5176.468263] kthread+0x1d1/0x200 [ 5176.469974] ? set_kthread_struct+0x70/0x70 [ 5176.471541] ret_from_fork+0x1f/0x30 [ 5180.058027] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 5180.061420] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5180.099668] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5180.106426] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5180.109509] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5180.112081] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5180.113765] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5180.120129] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5180.123222] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5180.127720] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5180.129951] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5180.132016] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5180.136331] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5180.149583] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5180.159200] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5180.163629] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5180.172535] Call Trace: [ 5180.176937] [ 5180.179797] ? show_regs.cold.9+0x22/0x2f [ 5180.183908] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5180.186577] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5180.192549] ? watchdog+0x40/0x40 [ 5180.197064] ? hrtimer_interrupt+0x128/0x2c0 [ 5180.201899] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5180.206643] ? apic_timer_interrupt+0xf/0x20 [ 5180.209434] [ 5180.211817] ? native_safe_halt+0x2e/0x50 [ 5180.221171] kvm_wait+0xcd/0xe0 [ 5180.222048] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5180.230030] do_raw_spin_lock+0x177/0x190 [ 5180.234832] _raw_spin_lock+0x21/0x30 [ 5180.237948] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5180.240900] process_one_work+0x2c8/0x700 [ 5180.242785] worker_thread+0x296/0x6e0 [ 5180.244036] ? rescuer_thread+0x570/0x570 [ 5180.246152] kthread+0x1d1/0x200 [ 5180.247665] ? set_kthread_struct+0x70/0x70 [ 5180.249391] ret_from_fork+0x1f/0x30 [ 5204.055041] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5204.063090] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5204.064173] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5204.088744] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5204.088763] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5204.088767] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5204.088788] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5204.088793] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5204.088800] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5204.088802] RDX: 000000000000260d RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5204.088805] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5204.088807] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5204.088809] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5204.088813] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5204.088815] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5204.088818] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5204.088827] Call Trace: [ 5204.088834] [ 5204.088842] ? show_regs.cold.9+0x22/0x2f [ 5204.088926] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5204.088934] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5204.088939] ? watchdog+0x40/0x40 [ 5204.088944] ? hrtimer_interrupt+0x128/0x2c0 [ 5204.088948] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5204.088961] ? apic_timer_interrupt+0xf/0x20 [ 5204.088966] [ 5204.088970] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5204.088976] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5204.088982] do_raw_spin_lock+0x177/0x190 [ 5204.088988] _raw_spin_lock+0x21/0x30 [ 5204.089049] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5204.089416] obd_mod_free+0x46/0x140 [ptlrpc] [ 5204.090061] mdc_commit_open+0x91/0xb0 [mdc] [ 5204.090100] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5204.090363] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5204.090558] after_reply+0xe2c/0x1780 [ptlrpc] [ 5204.090804] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5204.093855] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5204.093946] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5204.093955] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5204.093959] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5204.093976] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5204.093980] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5204.093986] RAX: 0000000000003aab RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5204.093989] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5204.093991] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5204.093993] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5204.093996] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5204.093999] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5204.094002] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5204.094005] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5204.094014] Call Trace: [ 5204.094021] [ 5204.094031] ? show_regs.cold.9+0x22/0x2f [ 5204.094045] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5204.094053] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5204.094057] ? watchdog+0x40/0x40 [ 5204.094062] ? hrtimer_interrupt+0x128/0x2c0 [ 5204.094066] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5204.094076] ? apic_timer_interrupt+0xf/0x20 [ 5204.094082] [ 5204.094084] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5204.094089] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5204.094094] do_raw_spin_lock+0x177/0x190 [ 5204.094098] _raw_spin_lock+0x21/0x30 [ 5204.094104] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5204.102385] ? __next_timer_interrupt+0x160/0x160 [ 5204.102415] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5204.110265] ? do_raw_spin_unlock+0x75/0x190 [ 5204.110294] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5204.110604] ? do_wait_intr+0xf0/0xf0 [ 5204.110614] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5204.117991] ? do_wait_intr+0xf0/0xf0 [ 5204.118019] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5204.118320] kthread+0x1d1/0x200 [ 5204.118327] ? set_kthread_struct+0x70/0x70 [ 5204.118331] ret_from_fork+0x1f/0x30 [ 5204.399767] mdc_reint+0x2d/0x110 [mdc] [ 5204.401991] mdc_unlink+0x658/0x870 [mdc] [ 5204.404560] lmv_unlink+0x34d/0xbd0 [lmv] [ 5204.407446] ll_unlink+0x24d/0x7f0 [lustre] [ 5204.417708] vfs_unlink+0x202/0x330 [ 5204.427109] do_unlinkat+0x2fa/0x460 [ 5204.431931] __x64_sys_unlink+0x37/0x50 [ 5204.433119] do_syscall_64+0xc1/0x440 [ 5204.435540] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5204.438046] RIP: 0033:0x7f33957fc10b [ 5204.438919] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5204.443216] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5204.445047] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5204.448015] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5204.450413] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5204.452412] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5204.454777] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5208.061004] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5208.063803] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5208.173053] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5208.191724] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5208.202647] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5208.212017] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5208.221281] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5208.250648] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5208.253273] RAX: 0000000000002d0e RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5208.273145] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5208.280284] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5208.293362] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5208.299760] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5208.309762] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5208.320564] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5208.331141] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5208.341002] Call Trace: [ 5208.344551] [ 5208.349617] ? show_regs.cold.9+0x22/0x2f [ 5208.354822] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5208.359875] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5208.368580] ? watchdog+0x40/0x40 [ 5208.375091] ? hrtimer_interrupt+0x128/0x2c0 [ 5208.384256] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5208.387322] ? apic_timer_interrupt+0xf/0x20 [ 5208.395853] [ 5208.403715] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5208.416295] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5208.431141] do_raw_spin_lock+0x177/0x190 [ 5208.438174] _raw_spin_lock+0x21/0x30 [ 5208.449691] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5208.451885] process_one_work+0x2c8/0x700 [ 5208.469233] worker_thread+0x296/0x6e0 [ 5208.475618] ? rescuer_thread+0x570/0x570 [ 5208.482634] kthread+0x1d1/0x200 [ 5208.488404] ? set_kthread_struct+0x70/0x70 [ 5208.493074] ret_from_fork+0x1f/0x30 [ 5218.093036] rcu: INFO: rcu_sched self-detected stall on CPU [ 5218.095717] rcu: 0-....: (1928573 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=587154 [ 5218.109546] (t=3120073 jiffies g=297885 q=15581) [ 5218.122720] NMI backtrace for cpu 0 [ 5218.126165] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5218.140109] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5218.146974] Call Trace: [ 5218.147837] [ 5218.149035] ? dump_stack+0xbb/0x10e [ 5218.154827] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 5218.156598] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 5218.160659] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 5218.166920] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 5218.171417] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 5218.176920] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 5218.180041] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 5218.184523] ? update_process_times+0x6d/0xc0 [ 5218.190095] ? tick_sched_handle.isra.18+0x27/0x90 [ 5218.197121] ? tick_sched_timer+0x53/0xb0 [ 5218.201791] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5218.206535] ? tick_sched_handle.isra.18+0x90/0x90 [ 5218.210117] ? hrtimer_interrupt+0x128/0x2c0 [ 5218.212957] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5218.218585] ? apic_timer_interrupt+0xf/0x20 [ 5218.220186] [ 5218.224927] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5218.228507] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5218.233734] ? do_raw_spin_lock+0x177/0x190 [ 5218.236709] ? _raw_spin_lock+0x21/0x30 [ 5218.239755] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5218.244638] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 5218.250841] ? mdc_commit_open+0x91/0xb0 [mdc] [ 5218.260441] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5218.265089] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5218.269878] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 5218.281768] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5218.287184] ? __next_timer_interrupt+0x160/0x160 [ 5218.294848] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5218.301072] ? do_wait_intr+0xf0/0xf0 [ 5218.303070] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5218.318465] ? kthread+0x1d1/0x200 [ 5218.323977] ? set_kthread_struct+0x70/0x70 [ 5218.328483] ? ret_from_fork+0x1f/0x30 [ 5232.059800] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5232.063748] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5232.103931] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5232.117006] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5232.131952] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5232.141883] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5232.174441] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 5232.182918] RAX: 0000000000007ab5 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5232.198433] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5232.204313] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5232.224165] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5232.233855] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5232.244742] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5232.258458] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5232.264026] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5232.276939] Call Trace: [ 5232.278305] [ 5232.280225] ? show_regs.cold.9+0x22/0x2f [ 5232.293576] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5232.312453] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5232.321556] ? watchdog+0x40/0x40 [ 5232.336380] ? hrtimer_interrupt+0x128/0x2c0 [ 5232.337900] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5232.346027] ? apic_timer_interrupt+0xf/0x20 [ 5232.347567] [ 5232.351700] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5232.366216] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5232.380073] do_raw_spin_lock+0x177/0x190 [ 5232.384944] _raw_spin_lock+0x21/0x30 [ 5232.391035] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5232.397440] ? do_raw_spin_unlock+0x75/0x190 [ 5232.402213] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5232.407718] ? do_wait_intr+0xf0/0xf0 [ 5232.409142] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5232.421259] mdc_reint+0x2d/0x110 [mdc] [ 5232.424256] mdc_unlink+0x658/0x870 [mdc] [ 5232.428151] lmv_unlink+0x34d/0xbd0 [lmv] [ 5232.431898] ll_unlink+0x24d/0x7f0 [lustre] [ 5232.440240] vfs_unlink+0x202/0x330 [ 5232.443692] do_unlinkat+0x2fa/0x460 [ 5232.448890] __x64_sys_unlink+0x37/0x50 [ 5232.452550] do_syscall_64+0xc1/0x440 [ 5232.455387] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5232.461899] RIP: 0033:0x7f33957fc10b [ 5232.469804] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5232.492350] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5232.503131] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5232.509696] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5232.514781] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5232.523822] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5232.530973] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5236.058010] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5236.062962] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5236.121158] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5236.127397] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5236.138446] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5236.142312] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5236.145225] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5236.153334] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5236.156359] RAX: 0000000000001a22 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5236.159859] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5236.163172] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5236.174087] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5236.176432] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5236.178843] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5236.181153] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5236.182807] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5236.184866] Call Trace: [ 5236.185648] [ 5236.186235] ? show_regs.cold.9+0x22/0x2f [ 5236.187510] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5236.188848] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5236.190473] ? watchdog+0x40/0x40 [ 5236.191794] ? hrtimer_interrupt+0x128/0x2c0 [ 5236.193401] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5236.196343] ? apic_timer_interrupt+0xf/0x20 [ 5236.198331] [ 5236.199812] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5236.204125] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5236.208436] do_raw_spin_lock+0x177/0x190 [ 5236.211490] _raw_spin_lock+0x21/0x30 [ 5236.216100] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5236.227239] process_one_work+0x2c8/0x700 [ 5236.229185] worker_thread+0x296/0x6e0 [ 5236.231048] ? rescuer_thread+0x570/0x570 [ 5236.232854] kthread+0x1d1/0x200 [ 5236.234739] ? set_kthread_struct+0x70/0x70 [ 5236.236949] ret_from_fork+0x1f/0x30 [ 5244.056671] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5244.067957] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5244.121872] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5244.135935] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5244.150541] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5244.158980] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5244.176987] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5244.184808] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5244.190754] RDX: 0000000000004c20 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5244.197506] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5244.205095] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5244.213274] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5244.221939] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5244.236166] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5244.242853] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5244.251558] Call Trace: [ 5244.252399] [ 5244.253011] ? show_regs.cold.9+0x22/0x2f [ 5244.256735] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5244.262595] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5244.267177] ? watchdog+0x40/0x40 [ 5244.273879] ? hrtimer_interrupt+0x128/0x2c0 [ 5244.276956] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5244.279593] ? apic_timer_interrupt+0xf/0x20 [ 5244.281995] [ 5244.283747] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5244.291627] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5244.296976] do_raw_spin_lock+0x177/0x190 [ 5244.302488] _raw_spin_lock+0x21/0x30 [ 5244.305534] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5244.318597] obd_mod_free+0x46/0x140 [ptlrpc] [ 5244.327305] mdc_commit_open+0x91/0xb0 [mdc] [ 5244.331559] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5244.336089] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5244.344513] after_reply+0xe2c/0x1780 [ptlrpc] [ 5244.348567] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5244.356856] ? __next_timer_interrupt+0x160/0x160 [ 5244.365723] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5244.373590] ? do_wait_intr+0xf0/0xf0 [ 5244.377133] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5244.379289] kthread+0x1d1/0x200 [ 5244.382966] ? set_kthread_struct+0x70/0x70 [ 5244.387162] ret_from_fork+0x1f/0x30 [ 5260.059021] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5260.063265] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5260.096394] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5260.106846] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5260.114326] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5260.116379] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5260.131413] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5260.135534] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5260.141831] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5260.145625] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5260.151025] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5260.153778] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5260.156642] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5260.166389] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5260.170727] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5260.174152] Call Trace: [ 5260.176319] [ 5260.178117] ? show_regs.cold.9+0x22/0x2f [ 5260.180987] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5260.187343] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5260.188881] ? watchdog+0x40/0x40 [ 5260.190440] ? hrtimer_interrupt+0x128/0x2c0 [ 5260.192374] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5260.196261] ? apic_timer_interrupt+0xf/0x20 [ 5260.199584] [ 5260.200796] ? native_safe_halt+0x2e/0x50 [ 5260.202185] kvm_wait+0xcd/0xe0 [ 5260.203271] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5260.208642] do_raw_spin_lock+0x177/0x190 [ 5260.211293] _raw_spin_lock+0x21/0x30 [ 5260.213755] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5260.218955] ? do_raw_spin_unlock+0x75/0x190 [ 5260.222828] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5260.226904] ? do_wait_intr+0xf0/0xf0 [ 5260.230441] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5260.232188] mdc_reint+0x2d/0x110 [mdc] [ 5260.234351] mdc_unlink+0x658/0x870 [mdc] [ 5260.236627] lmv_unlink+0x34d/0xbd0 [lmv] [ 5260.242284] ll_unlink+0x24d/0x7f0 [lustre] [ 5260.244480] vfs_unlink+0x202/0x330 [ 5260.245833] do_unlinkat+0x2fa/0x460 [ 5260.250873] __x64_sys_unlink+0x37/0x50 [ 5260.254256] do_syscall_64+0xc1/0x440 [ 5260.256424] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5260.263439] RIP: 0033:0x7f33957fc10b [ 5260.269210] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5260.297508] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5260.304944] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5260.308437] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5260.311971] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5260.318379] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5260.322421] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5264.058023] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5264.068396] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5264.114620] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5264.123197] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5264.133438] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5264.140590] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5264.144875] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5264.169901] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5264.181671] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5264.192386] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5264.201574] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5264.210755] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5264.221256] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5264.233227] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5264.246530] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5264.255622] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5264.265218] Call Trace: [ 5264.266741] [ 5264.269153] ? show_regs.cold.9+0x22/0x2f [ 5264.271718] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5264.281234] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5264.286258] ? watchdog+0x40/0x40 [ 5264.290711] ? hrtimer_interrupt+0x128/0x2c0 [ 5264.294526] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5264.297805] ? apic_timer_interrupt+0xf/0x20 [ 5264.306317] [ 5264.309658] ? native_safe_halt+0x2e/0x50 [ 5264.316849] kvm_wait+0xcd/0xe0 [ 5264.319274] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5264.322399] do_raw_spin_lock+0x177/0x190 [ 5264.326629] _raw_spin_lock+0x21/0x30 [ 5264.332176] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5264.335270] process_one_work+0x2c8/0x700 [ 5264.337546] worker_thread+0x296/0x6e0 [ 5264.343198] ? rescuer_thread+0x570/0x570 [ 5264.344645] kthread+0x1d1/0x200 [ 5264.348747] ? set_kthread_struct+0x70/0x70 [ 5264.356078] ret_from_fork+0x1f/0x30 [ 5272.054515] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5272.061050] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5272.116649] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5272.124551] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5272.129479] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5272.134505] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5272.155237] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5272.158297] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5272.161206] RDX: 00000000000000e6 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5272.170705] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5272.180106] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5272.182381] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5272.189434] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5272.192044] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5272.194377] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5272.197431] Call Trace: [ 5272.198354] [ 5272.199175] ? show_regs.cold.9+0x22/0x2f [ 5272.200580] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5272.201816] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5272.203501] ? watchdog+0x40/0x40 [ 5272.205073] ? hrtimer_interrupt+0x128/0x2c0 [ 5272.207486] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5272.211310] ? apic_timer_interrupt+0xf/0x20 [ 5272.213341] [ 5272.214907] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5272.218939] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5272.220497] do_raw_spin_lock+0x177/0x190 [ 5272.225436] _raw_spin_lock+0x21/0x30 [ 5272.228543] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5272.241766] obd_mod_free+0x46/0x140 [ptlrpc] [ 5272.250689] mdc_commit_open+0x91/0xb0 [mdc] [ 5272.258063] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5272.269091] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5272.279586] after_reply+0xe2c/0x1780 [ptlrpc] [ 5272.281916] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5272.295473] ? __next_timer_interrupt+0x160/0x160 [ 5272.304786] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5272.312649] ? do_wait_intr+0xf0/0xf0 [ 5272.318579] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5272.326227] kthread+0x1d1/0x200 [ 5272.328578] ? set_kthread_struct+0x70/0x70 [ 5272.336026] ret_from_fork+0x1f/0x30 [ 5288.061697] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5288.070326] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5288.116268] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5288.127094] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5288.132134] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5288.137347] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5288.156729] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5288.163812] RAX: 0000000000007b08 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5288.172282] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5288.180005] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5288.185189] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5288.192917] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5288.198890] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5288.208396] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5288.211166] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5288.214910] Call Trace: [ 5288.216825] [ 5288.217733] ? show_regs.cold.9+0x22/0x2f [ 5288.219421] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5288.221466] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5288.225362] ? watchdog+0x40/0x40 [ 5288.227455] ? hrtimer_interrupt+0x128/0x2c0 [ 5288.230345] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5288.233319] ? apic_timer_interrupt+0xf/0x20 [ 5288.235378] [ 5288.238212] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5288.242765] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5288.246035] do_raw_spin_lock+0x177/0x190 [ 5288.248929] _raw_spin_lock+0x21/0x30 [ 5288.250398] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5288.257160] ? do_raw_spin_unlock+0x75/0x190 [ 5288.260347] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5288.264805] ? do_wait_intr+0xf0/0xf0 [ 5288.267078] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5288.269972] mdc_reint+0x2d/0x110 [mdc] [ 5288.272343] mdc_unlink+0x658/0x870 [mdc] [ 5288.274469] lmv_unlink+0x34d/0xbd0 [lmv] [ 5288.280312] ll_unlink+0x24d/0x7f0 [lustre] [ 5288.282711] vfs_unlink+0x202/0x330 [ 5288.284295] do_unlinkat+0x2fa/0x460 [ 5288.285728] __x64_sys_unlink+0x37/0x50 [ 5288.288140] do_syscall_64+0xc1/0x440 [ 5288.291304] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5288.295962] RIP: 0033:0x7f33957fc10b [ 5288.306278] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5288.325355] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5288.340340] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5288.349449] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5288.359876] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5288.368748] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5288.379166] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5292.058136] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5292.064120] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5292.094180] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5292.100145] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5292.109590] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5292.117203] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5292.130005] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5292.148010] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5292.154971] RAX: 0000000000005487 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5292.161118] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5292.168278] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5292.171489] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5292.174288] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5292.180602] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5292.187373] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5292.192023] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5292.196383] Call Trace: [ 5292.201367] [ 5292.204121] ? show_regs.cold.9+0x22/0x2f [ 5292.207225] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5292.209364] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5292.213352] ? watchdog+0x40/0x40 [ 5292.216034] ? hrtimer_interrupt+0x128/0x2c0 [ 5292.218178] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5292.220537] ? apic_timer_interrupt+0xf/0x20 [ 5292.222180] [ 5292.223250] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5292.225841] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5292.229251] do_raw_spin_lock+0x177/0x190 [ 5292.231739] _raw_spin_lock+0x21/0x30 [ 5292.233568] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5292.235966] process_one_work+0x2c8/0x700 [ 5292.238467] worker_thread+0x296/0x6e0 [ 5292.240257] ? rescuer_thread+0x570/0x570 [ 5292.243016] kthread+0x1d1/0x200 [ 5292.244407] ? set_kthread_struct+0x70/0x70 [ 5292.246903] ret_from_fork+0x1f/0x30 [ 5300.054935] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 5300.067451] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5300.100904] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5300.106688] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5300.111038] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5300.113593] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5300.130863] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5300.135883] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5300.139452] RDX: 00000000000076cc RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5300.142564] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5300.148813] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5300.154630] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5300.163243] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5300.170012] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5300.174210] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5300.178372] Call Trace: [ 5300.179706] [ 5300.180505] ? show_regs.cold.9+0x22/0x2f [ 5300.182595] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5300.185059] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5300.187336] ? watchdog+0x40/0x40 [ 5300.189621] ? hrtimer_interrupt+0x128/0x2c0 [ 5300.192862] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5300.199635] ? apic_timer_interrupt+0xf/0x20 [ 5300.202503] [ 5300.203563] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5300.212143] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5300.216042] do_raw_spin_lock+0x177/0x190 [ 5300.218937] _raw_spin_lock+0x21/0x30 [ 5300.221487] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5300.224920] obd_mod_free+0x46/0x140 [ptlrpc] [ 5300.227459] mdc_commit_open+0x91/0xb0 [mdc] [ 5300.230735] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5300.234642] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5300.239250] after_reply+0xe2c/0x1780 [ptlrpc] [ 5300.241473] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5300.244226] ? __next_timer_interrupt+0x160/0x160 [ 5300.248567] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5300.253071] ? do_wait_intr+0xf0/0xf0 [ 5300.255791] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5300.259802] kthread+0x1d1/0x200 [ 5300.267203] ? set_kthread_struct+0x70/0x70 [ 5300.270251] ret_from_fork+0x1f/0x30 [ 5316.059149] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5316.063321] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5316.097203] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5316.105205] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5316.110513] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5316.112360] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5316.120585] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5316.124163] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5316.127521] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5316.130479] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5316.133601] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5316.136814] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5316.146028] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5316.156647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5316.160128] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5316.163685] Call Trace: [ 5316.164721] [ 5316.166323] ? show_regs.cold.9+0x22/0x2f [ 5316.168573] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5316.171087] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5316.174247] ? watchdog+0x40/0x40 [ 5316.176641] ? hrtimer_interrupt+0x128/0x2c0 [ 5316.186844] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5316.193190] ? apic_timer_interrupt+0xf/0x20 [ 5316.199778] [ 5316.201450] ? native_safe_halt+0x2e/0x50 [ 5316.204716] kvm_wait+0xcd/0xe0 [ 5316.210177] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5316.212834] do_raw_spin_lock+0x177/0x190 [ 5316.214276] _raw_spin_lock+0x21/0x30 [ 5316.215552] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5316.226053] ? do_raw_spin_unlock+0x75/0x190 [ 5316.227902] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5316.229907] ? do_wait_intr+0xf0/0xf0 [ 5316.231359] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5316.235719] mdc_reint+0x2d/0x110 [mdc] [ 5316.237986] mdc_unlink+0x658/0x870 [mdc] [ 5316.239595] lmv_unlink+0x34d/0xbd0 [lmv] [ 5316.242685] ll_unlink+0x24d/0x7f0 [lustre] [ 5316.244518] vfs_unlink+0x202/0x330 [ 5316.245867] do_unlinkat+0x2fa/0x460 [ 5316.251399] __x64_sys_unlink+0x37/0x50 [ 5316.253334] do_syscall_64+0xc1/0x440 [ 5316.256721] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5316.261014] RIP: 0033:0x7f33957fc10b [ 5316.264428] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5316.276284] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5316.281284] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5316.285565] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5316.290556] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5316.297222] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5316.301067] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5320.058026] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5320.062821] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5320.095991] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5320.106812] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5320.116496] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5320.123309] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5320.125247] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5320.140949] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5320.144045] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5320.160432] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5320.170721] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5320.174258] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5320.183054] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5320.185512] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5320.188720] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5320.191016] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5320.194232] Call Trace: [ 5320.195176] [ 5320.195755] ? show_regs.cold.9+0x22/0x2f [ 5320.196923] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5320.198232] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5320.199714] ? watchdog+0x40/0x40 [ 5320.200803] ? hrtimer_interrupt+0x128/0x2c0 [ 5320.202095] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5320.203517] ? apic_timer_interrupt+0xf/0x20 [ 5320.204913] [ 5320.205510] ? native_safe_halt+0x2e/0x50 [ 5320.206811] kvm_wait+0xcd/0xe0 [ 5320.207826] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5320.209379] do_raw_spin_lock+0x177/0x190 [ 5320.210626] _raw_spin_lock+0x21/0x30 [ 5320.212045] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5320.213774] process_one_work+0x2c8/0x700 [ 5320.214887] worker_thread+0x296/0x6e0 [ 5320.215832] ? rescuer_thread+0x570/0x570 [ 5320.216946] kthread+0x1d1/0x200 [ 5320.218040] ? set_kthread_struct+0x70/0x70 [ 5320.219452] ret_from_fork+0x1f/0x30 [ 5328.060494] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 5328.069416] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5328.113738] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5328.123298] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5328.129205] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5328.133327] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5328.146714] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5328.150886] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5328.155576] RDX: 0000000000004af4 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5328.159816] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5328.165093] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5328.171072] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5328.175123] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5328.180278] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5328.184131] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5328.189290] Call Trace: [ 5328.193712] [ 5328.198444] ? show_regs.cold.9+0x22/0x2f [ 5328.206088] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5328.213341] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5328.217728] ? watchdog+0x40/0x40 [ 5328.232547] ? hrtimer_interrupt+0x128/0x2c0 [ 5328.238763] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5328.247616] ? apic_timer_interrupt+0xf/0x20 [ 5328.252894] [ 5328.256521] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5328.267485] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5328.273547] do_raw_spin_lock+0x177/0x190 [ 5328.279789] _raw_spin_lock+0x21/0x30 [ 5328.283996] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5328.293707] obd_mod_free+0x46/0x140 [ptlrpc] [ 5328.298468] mdc_commit_open+0x91/0xb0 [mdc] [ 5328.307306] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5328.316216] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5328.321945] after_reply+0xe2c/0x1780 [ptlrpc] [ 5328.326631] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5328.330051] ? __next_timer_interrupt+0x160/0x160 [ 5328.335331] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5328.340608] ? do_wait_intr+0xf0/0xf0 [ 5328.343597] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5328.350297] kthread+0x1d1/0x200 [ 5328.355621] ? set_kthread_struct+0x70/0x70 [ 5328.360739] ret_from_fork+0x1f/0x30 [ 5344.059056] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 5344.065344] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5344.103733] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5344.109630] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5344.114189] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5344.117325] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5344.127788] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5344.134076] RAX: 0000000000000a8a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5344.139564] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5344.144358] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5344.151883] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5344.156328] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5344.159214] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5344.163525] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5344.166216] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5344.169834] Call Trace: [ 5344.171294] [ 5344.176076] ? show_regs.cold.9+0x22/0x2f [ 5344.177566] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5344.181229] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5344.185131] ? watchdog+0x40/0x40 [ 5344.187254] ? hrtimer_interrupt+0x128/0x2c0 [ 5344.190370] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5344.194686] ? apic_timer_interrupt+0xf/0x20 [ 5344.197531] [ 5344.198586] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5344.202551] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5344.206431] do_raw_spin_lock+0x177/0x190 [ 5344.208447] _raw_spin_lock+0x21/0x30 [ 5344.209787] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5344.212389] ? do_raw_spin_unlock+0x75/0x190 [ 5344.214271] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5344.217205] ? do_wait_intr+0xf0/0xf0 [ 5344.219177] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5344.222017] mdc_reint+0x2d/0x110 [mdc] [ 5344.223723] mdc_unlink+0x658/0x870 [mdc] [ 5344.225589] lmv_unlink+0x34d/0xbd0 [lmv] [ 5344.227387] ll_unlink+0x24d/0x7f0 [lustre] [ 5344.229412] vfs_unlink+0x202/0x330 [ 5344.230616] do_unlinkat+0x2fa/0x460 [ 5344.231761] __x64_sys_unlink+0x37/0x50 [ 5344.233806] do_syscall_64+0xc1/0x440 [ 5344.236323] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5344.239096] RIP: 0033:0x7f33957fc10b [ 5344.242250] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5344.259573] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5344.264767] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5344.270796] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5344.277492] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5344.282576] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5344.286294] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5348.060671] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5348.067165] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5348.106082] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5348.116818] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5348.126951] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5348.132896] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5348.138048] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5348.153134] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000216 ORIG_RAX: ffffffffffffff13 [ 5348.159562] RAX: 0000000000007b86 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5348.166929] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5348.171274] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5348.177477] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5348.183206] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5348.189155] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5348.196586] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5348.201751] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5348.207795] Call Trace: [ 5348.209855] [ 5348.211757] ? show_regs.cold.9+0x22/0x2f [ 5348.215585] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5348.220274] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5348.224294] ? watchdog+0x40/0x40 [ 5348.227696] ? hrtimer_interrupt+0x128/0x2c0 [ 5348.230493] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5348.235686] ? apic_timer_interrupt+0xf/0x20 [ 5348.242691] [ 5348.244013] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5348.249602] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5348.257236] do_raw_spin_lock+0x177/0x190 [ 5348.263912] _raw_spin_lock+0x21/0x30 [ 5348.270401] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5348.272488] process_one_work+0x2c8/0x700 [ 5348.287785] worker_thread+0x296/0x6e0 [ 5348.288922] ? rescuer_thread+0x570/0x570 [ 5348.300203] kthread+0x1d1/0x200 [ 5348.311731] ? set_kthread_struct+0x70/0x70 [ 5348.314924] ret_from_fork+0x1f/0x30 [ 5356.053021] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 5356.060483] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5356.093974] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5356.098989] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5356.102538] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5356.103971] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5356.112129] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5356.115810] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5356.119640] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5356.123110] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5356.126043] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5356.131111] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5356.136363] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5356.144638] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5356.150610] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5356.158952] Call Trace: [ 5356.160592] [ 5356.162003] ? show_regs.cold.9+0x22/0x2f [ 5356.166424] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5356.172237] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5356.177042] ? watchdog+0x40/0x40 [ 5356.180109] ? hrtimer_interrupt+0x128/0x2c0 [ 5356.183923] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5356.186015] ? apic_timer_interrupt+0xf/0x20 [ 5356.192879] [ 5356.195312] ? native_safe_halt+0x2e/0x50 [ 5356.199503] kvm_wait+0xcd/0xe0 [ 5356.201871] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5356.206979] do_raw_spin_lock+0x177/0x190 [ 5356.212896] _raw_spin_lock+0x21/0x30 [ 5356.218592] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5356.231062] obd_mod_free+0x46/0x140 [ptlrpc] [ 5356.236106] mdc_commit_open+0x91/0xb0 [mdc] [ 5356.240951] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5356.244765] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5356.250601] after_reply+0xe2c/0x1780 [ptlrpc] [ 5356.261044] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5356.268460] ? __next_timer_interrupt+0x160/0x160 [ 5356.273470] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5356.276949] ? do_wait_intr+0xf0/0xf0 [ 5356.283884] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5356.290221] kthread+0x1d1/0x200 [ 5356.294449] ? set_kthread_struct+0x70/0x70 [ 5356.298430] ret_from_fork+0x1f/0x30 [ 5372.059033] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 5372.062759] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5372.106073] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5372.116515] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5372.125452] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5372.129004] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5372.147243] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5372.153702] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5372.162784] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5372.167943] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5372.180278] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5372.184579] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5372.189837] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5372.204599] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5372.210027] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5372.216165] Call Trace: [ 5372.218980] [ 5372.220540] ? show_regs.cold.9+0x22/0x2f [ 5372.222793] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5372.225627] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5372.232781] ? watchdog+0x40/0x40 [ 5372.234656] ? hrtimer_interrupt+0x128/0x2c0 [ 5372.241774] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5372.246641] ? apic_timer_interrupt+0xf/0x20 [ 5372.249759] [ 5372.251785] ? native_safe_halt+0x2e/0x50 [ 5372.254932] kvm_wait+0xcd/0xe0 [ 5372.256915] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5372.262738] do_raw_spin_lock+0x177/0x190 [ 5372.270779] _raw_spin_lock+0x21/0x30 [ 5372.272766] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5372.276570] ? do_raw_spin_unlock+0x75/0x190 [ 5372.285183] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5372.292709] ? do_wait_intr+0xf0/0xf0 [ 5372.296215] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5372.300095] mdc_reint+0x2d/0x110 [mdc] [ 5372.302584] mdc_unlink+0x658/0x870 [mdc] [ 5372.306710] lmv_unlink+0x34d/0xbd0 [lmv] [ 5372.319945] ll_unlink+0x24d/0x7f0 [lustre] [ 5372.327371] vfs_unlink+0x202/0x330 [ 5372.328451] do_unlinkat+0x2fa/0x460 [ 5372.329272] __x64_sys_unlink+0x37/0x50 [ 5372.332201] do_syscall_64+0xc1/0x440 [ 5372.334437] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5372.336831] RIP: 0033:0x7f33957fc10b [ 5372.338580] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5372.354798] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5372.357546] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5372.361688] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5372.364361] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5372.366750] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5372.369634] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5376.059247] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5376.068994] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5376.127931] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5376.138002] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5376.153381] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5376.163782] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5376.165200] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5376.182950] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5376.188102] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5376.194001] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5376.204531] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5376.208029] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5376.211379] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5376.219349] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5376.226939] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5376.230851] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5376.235720] Call Trace: [ 5376.236937] [ 5376.237910] ? show_regs.cold.9+0x22/0x2f [ 5376.241731] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5376.245412] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5376.247170] ? watchdog+0x40/0x40 [ 5376.250245] ? hrtimer_interrupt+0x128/0x2c0 [ 5376.255371] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5376.262250] ? apic_timer_interrupt+0xf/0x20 [ 5376.263932] [ 5376.266028] ? native_safe_halt+0x2e/0x50 [ 5376.270317] kvm_wait+0xcd/0xe0 [ 5376.272071] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5376.274593] do_raw_spin_lock+0x177/0x190 [ 5376.276399] _raw_spin_lock+0x21/0x30 [ 5376.278044] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5376.283613] process_one_work+0x2c8/0x700 [ 5376.288570] worker_thread+0x296/0x6e0 [ 5376.289893] ? rescuer_thread+0x570/0x570 [ 5376.297046] kthread+0x1d1/0x200 [ 5376.299002] ? set_kthread_struct+0x70/0x70 [ 5376.301518] ret_from_fork+0x1f/0x30 [ 5384.053054] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ptlrpcd_rcv:2387] [ 5384.056578] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5384.101576] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5384.115405] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5384.126818] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5384.131989] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5384.150510] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5384.159926] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5384.169077] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5384.175473] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5384.183577] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5384.190829] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5384.196384] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5384.203965] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5384.210373] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5384.217796] Call Trace: [ 5384.220753] [ 5384.223374] ? show_regs.cold.9+0x22/0x2f [ 5384.226928] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5384.233351] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5384.237545] ? watchdog+0x40/0x40 [ 5384.240656] ? hrtimer_interrupt+0x128/0x2c0 [ 5384.245379] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5384.248931] ? apic_timer_interrupt+0xf/0x20 [ 5384.252267] [ 5384.253507] ? native_safe_halt+0x2e/0x50 [ 5384.255961] kvm_wait+0xcd/0xe0 [ 5384.260382] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5384.266785] do_raw_spin_lock+0x177/0x190 [ 5384.271340] _raw_spin_lock+0x21/0x30 [ 5384.280169] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5384.287102] obd_mod_free+0x46/0x140 [ptlrpc] [ 5384.293142] mdc_commit_open+0x91/0xb0 [mdc] [ 5384.300128] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5384.306570] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5384.316453] after_reply+0xe2c/0x1780 [ptlrpc] [ 5384.322495] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5384.327457] ? __next_timer_interrupt+0x160/0x160 [ 5384.329776] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5384.334826] ? do_wait_intr+0xf0/0xf0 [ 5384.336955] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5384.342549] kthread+0x1d1/0x200 [ 5384.346797] ? set_kthread_struct+0x70/0x70 [ 5384.350770] ret_from_fork+0x1f/0x30 [ 5398.096042] rcu: INFO: rcu_sched self-detected stall on CPU [ 5398.098646] rcu: 0-....: (2028699 ticks this GP) idle=6d2/1/0x4000000000000002 softirq=141607/141607 fqs=619430 [ 5398.106582] (t=3300076 jiffies g=297885 q=16420) [ 5398.112635] NMI backtrace for cpu 0 [ 5398.114472] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5398.122196] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5398.128674] Call Trace: [ 5398.130458] [ 5398.131537] ? dump_stack+0xbb/0x10e [ 5398.133620] ? lapic_can_unplug_cpu.cold.28+0x16/0x55 [ 5398.136403] ? nmi_cpu_backtrace.cold.8+0x24/0xa2 [ 5398.138810] ? lapic_can_unplug_cpu.cold.28+0x55/0x55 [ 5398.141245] ? nmi_trigger_cpumask_backtrace+0x1d7/0x1f1 [ 5398.144178] ? arch_trigger_cpumask_backtrace+0x1d/0x30 [ 5398.146570] ? rcu_dump_cpu_stacks+0x11b/0x171 [ 5398.148638] ? rcu_sched_clock_irq.cold.51+0xdb/0x307 [ 5398.151237] ? update_process_times+0x6d/0xc0 [ 5398.153557] ? tick_sched_handle.isra.18+0x27/0x90 [ 5398.155716] ? tick_sched_timer+0x53/0xb0 [ 5398.157681] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5398.159804] ? tick_sched_handle.isra.18+0x90/0x90 [ 5398.162039] ? hrtimer_interrupt+0x128/0x2c0 [ 5398.163936] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5398.166066] ? apic_timer_interrupt+0xf/0x20 [ 5398.168113] [ 5398.169128] ? native_safe_halt+0x2e/0x50 [ 5398.170820] ? kvm_wait+0xcd/0xe0 [ 5398.172455] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5398.174613] ? do_raw_spin_lock+0x177/0x190 [ 5398.176577] ? _raw_spin_lock+0x21/0x30 [ 5398.178282] ? __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5398.180845] ? obd_mod_free+0x46/0x140 [ptlrpc] [ 5398.183258] ? mdc_commit_open+0x91/0xb0 [mdc] [ 5398.185285] ? ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5398.188832] ? ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5398.192681] ? after_reply+0xe2c/0x1780 [ptlrpc] [ 5398.195454] ? ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5398.199143] ? __next_timer_interrupt+0x160/0x160 [ 5398.203193] ? ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5398.206142] ? do_wait_intr+0xf0/0xf0 [ 5398.208080] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5398.212529] ? kthread+0x1d1/0x200 [ 5398.214936] ? set_kthread_struct+0x70/0x70 [ 5398.216651] ? ret_from_fork+0x1f/0x30 [ 5400.059033] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 5400.063078] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5400.089743] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5400.098064] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5400.104867] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5400.108013] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5400.123839] RSP: 0018:ffff985380d7fb98 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5400.133441] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5400.138221] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5400.143305] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5400.147692] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5400.151528] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5400.156159] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5400.161298] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5400.164585] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5400.167674] Call Trace: [ 5400.169408] [ 5400.170802] ? show_regs.cold.9+0x22/0x2f [ 5400.172892] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5400.175102] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5400.177212] ? watchdog+0x40/0x40 [ 5400.179558] ? hrtimer_interrupt+0x128/0x2c0 [ 5400.182258] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5400.184819] ? apic_timer_interrupt+0xf/0x20 [ 5400.186860] [ 5400.187839] ? native_safe_halt+0x2e/0x50 [ 5400.190546] kvm_wait+0xcd/0xe0 [ 5400.192537] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5400.195631] do_raw_spin_lock+0x177/0x190 [ 5400.197891] _raw_spin_lock+0x21/0x30 [ 5400.199553] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5400.204695] ? do_raw_spin_unlock+0x75/0x190 [ 5400.207881] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5400.211205] ? do_wait_intr+0xf0/0xf0 [ 5400.212985] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5400.215792] mdc_reint+0x2d/0x110 [mdc] [ 5400.217579] mdc_unlink+0x658/0x870 [mdc] [ 5400.219975] lmv_unlink+0x34d/0xbd0 [lmv] [ 5400.222496] ll_unlink+0x24d/0x7f0 [lustre] [ 5400.225045] vfs_unlink+0x202/0x330 [ 5400.226844] do_unlinkat+0x2fa/0x460 [ 5400.229810] __x64_sys_unlink+0x37/0x50 [ 5400.232862] do_syscall_64+0xc1/0x440 [ 5400.236182] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5400.238789] RIP: 0033:0x7f33957fc10b [ 5400.240338] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5400.251675] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5400.257425] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5400.260895] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5400.265245] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5400.270400] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5400.273898] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5404.058025] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5404.064507] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5404.103880] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5404.117336] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5404.129849] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5404.136682] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5404.139993] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5404.159910] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5404.170381] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5404.175401] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5404.182440] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5404.187042] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5404.193814] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5404.198089] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5404.204067] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5404.207573] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5404.210062] Call Trace: [ 5404.211410] [ 5404.212796] ? show_regs.cold.9+0x22/0x2f [ 5404.217507] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5404.226841] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5404.230684] ? watchdog+0x40/0x40 [ 5404.235186] ? hrtimer_interrupt+0x128/0x2c0 [ 5404.237654] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5404.242802] ? apic_timer_interrupt+0xf/0x20 [ 5404.246972] [ 5404.247995] ? native_safe_halt+0x2e/0x50 [ 5404.252000] kvm_wait+0xcd/0xe0 [ 5404.253870] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5404.259551] do_raw_spin_lock+0x177/0x190 [ 5404.262438] _raw_spin_lock+0x21/0x30 [ 5404.264614] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5404.271575] process_one_work+0x2c8/0x700 [ 5404.273044] worker_thread+0x296/0x6e0 [ 5404.277400] ? rescuer_thread+0x570/0x570 [ 5404.281531] kthread+0x1d1/0x200 [ 5404.287643] ? set_kthread_struct+0x70/0x70 [ 5404.292174] ret_from_fork+0x1f/0x30 [ 5424.053036] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5424.056253] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5424.078534] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5424.086041] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5424.090080] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5424.092166] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5424.107121] RSP: 0018:ffff9853816cfbb8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5424.109983] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5424.112870] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5424.116087] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5424.119168] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5424.121851] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5424.124726] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5424.128111] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5424.130928] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5424.138683] Call Trace: [ 5424.139880] [ 5424.144203] ? show_regs.cold.9+0x22/0x2f [ 5424.146229] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5424.148668] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5424.150391] ? watchdog+0x40/0x40 [ 5424.154481] ? hrtimer_interrupt+0x128/0x2c0 [ 5424.156286] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5424.158636] ? apic_timer_interrupt+0xf/0x20 [ 5424.160213] [ 5424.161041] ? native_safe_halt+0x2e/0x50 [ 5424.162703] kvm_wait+0xcd/0xe0 [ 5424.164423] __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5424.171057] do_raw_spin_lock+0x177/0x190 [ 5424.173293] _raw_spin_lock+0x21/0x30 [ 5424.174571] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5424.177289] obd_mod_free+0x46/0x140 [ptlrpc] [ 5424.179218] mdc_commit_open+0x91/0xb0 [mdc] [ 5424.183151] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5424.186190] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5424.189156] after_reply+0xe2c/0x1780 [ptlrpc] [ 5424.191388] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5424.193871] ? __next_timer_interrupt+0x160/0x160 [ 5424.197594] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5424.199600] ? do_wait_intr+0xf0/0xf0 [ 5424.200597] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5424.203535] kthread+0x1d1/0x200 [ 5424.205343] ? set_kthread_struct+0x70/0x70 [ 5424.207616] ret_from_fork+0x1f/0x30 [ 5428.068119] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [writemany:37219] [ 5428.079583] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5428.132387] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5428.141169] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5428.146839] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5428.149411] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5428.155995] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5428.160663] RAX: 000000000000612a RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5428.170240] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5428.176641] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5428.185414] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5428.196989] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5428.205677] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5428.213239] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5428.220830] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5428.226129] Call Trace: [ 5428.227038] [ 5428.227775] ? show_regs.cold.9+0x22/0x2f [ 5428.229589] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5428.239480] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5428.243554] ? watchdog+0x40/0x40 [ 5428.244613] ? hrtimer_interrupt+0x128/0x2c0 [ 5428.249542] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5428.251016] ? apic_timer_interrupt+0xf/0x20 [ 5428.257456] [ 5428.259248] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5428.265332] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5428.268334] do_raw_spin_lock+0x177/0x190 [ 5428.272754] _raw_spin_lock+0x21/0x30 [ 5428.273780] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5428.278653] ? do_raw_spin_unlock+0x75/0x190 [ 5428.281920] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5428.284570] ? do_wait_intr+0xf0/0xf0 [ 5428.288886] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5428.295131] mdc_reint+0x2d/0x110 [mdc] [ 5428.297501] mdc_unlink+0x658/0x870 [mdc] [ 5428.301426] lmv_unlink+0x34d/0xbd0 [lmv] [ 5428.302936] ll_unlink+0x24d/0x7f0 [lustre] [ 5428.305859] vfs_unlink+0x202/0x330 [ 5428.307495] do_unlinkat+0x2fa/0x460 [ 5428.309511] __x64_sys_unlink+0x37/0x50 [ 5428.311287] do_syscall_64+0xc1/0x440 [ 5428.312537] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5428.317444] RIP: 0033:0x7f33957fc10b [ 5428.318422] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5428.331742] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5428.337021] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5428.344084] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5428.347363] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5428.354150] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5428.357608] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5432.058114] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u8:2:37] [ 5432.063341] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5432.092768] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5432.099957] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5432.104532] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5432.108155] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5432.111092] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5432.121727] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5432.124893] RAX: 000000000000140f RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5432.128734] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5432.132576] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5432.136461] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5432.142727] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5432.147568] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5432.156611] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5432.162715] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5432.166836] Call Trace: [ 5432.168103] [ 5432.169483] ? show_regs.cold.9+0x22/0x2f [ 5432.171335] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5432.173120] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5432.175308] ? watchdog+0x40/0x40 [ 5432.176902] ? hrtimer_interrupt+0x128/0x2c0 [ 5432.178705] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5432.180751] ? apic_timer_interrupt+0xf/0x20 [ 5432.182273] [ 5432.183310] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5432.185460] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5432.187880] do_raw_spin_lock+0x177/0x190 [ 5432.189777] _raw_spin_lock+0x21/0x30 [ 5432.192131] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5432.194797] process_one_work+0x2c8/0x700 [ 5432.196583] worker_thread+0x296/0x6e0 [ 5432.198566] ? rescuer_thread+0x570/0x570 [ 5432.200786] kthread+0x1d1/0x200 [ 5432.201959] ? set_kthread_struct+0x70/0x70 [ 5432.204278] ret_from_fork+0x1f/0x30 [ 5452.060948] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5452.068129] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5452.118671] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5452.129117] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5452.140102] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5452.145959] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5452.159994] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5452.166995] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5452.175635] RDX: 0000000000002e68 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5452.182037] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5452.186392] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5452.190645] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5452.194732] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5452.199894] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5452.203651] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5452.213693] Call Trace: [ 5452.216733] [ 5452.219028] ? show_regs.cold.9+0x22/0x2f [ 5452.223100] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5452.230928] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5452.240583] ? watchdog+0x40/0x40 [ 5452.246484] ? hrtimer_interrupt+0x128/0x2c0 [ 5452.247983] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5452.253817] ? apic_timer_interrupt+0xf/0x20 [ 5452.255211] [ 5452.255887] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5452.264895] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5452.269151] do_raw_spin_lock+0x177/0x190 [ 5452.275351] _raw_spin_lock+0x21/0x30 [ 5452.281043] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5452.283073] obd_mod_free+0x46/0x140 [ptlrpc] [ 5452.288080] mdc_commit_open+0x91/0xb0 [mdc] [ 5452.290680] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5452.293399] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5452.295877] after_reply+0xe2c/0x1780 [ptlrpc] [ 5452.298726] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5452.302072] ? __next_timer_interrupt+0x160/0x160 [ 5452.308109] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5452.311791] ? do_wait_intr+0xf0/0xf0 [ 5452.315057] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5452.322180] kthread+0x1d1/0x200 [ 5452.324634] ? set_kthread_struct+0x70/0x70 [ 5452.326598] ret_from_fork+0x1f/0x30 [ 5456.061467] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5456.069373] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5456.107763] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5456.114597] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5456.121468] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5456.125848] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5456.137369] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5456.141328] RAX: 0000000000005b84 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5456.144956] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5456.150989] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5456.155599] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5456.160569] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5456.165514] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5456.171232] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5456.174372] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5456.180984] Call Trace: [ 5456.182966] [ 5456.184013] ? show_regs.cold.9+0x22/0x2f [ 5456.188360] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5456.191017] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5456.193123] ? watchdog+0x40/0x40 [ 5456.196030] ? hrtimer_interrupt+0x128/0x2c0 [ 5456.202104] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5456.204873] ? apic_timer_interrupt+0xf/0x20 [ 5456.209063] [ 5456.211544] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5456.213751] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5456.215474] do_raw_spin_lock+0x177/0x190 [ 5456.219745] _raw_spin_lock+0x21/0x30 [ 5456.224144] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5456.232471] ? do_raw_spin_unlock+0x75/0x190 [ 5456.234140] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5456.241328] ? do_wait_intr+0xf0/0xf0 [ 5456.246364] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5456.250281] mdc_reint+0x2d/0x110 [mdc] [ 5456.256334] mdc_unlink+0x658/0x870 [mdc] [ 5456.258521] lmv_unlink+0x34d/0xbd0 [lmv] [ 5456.260266] ll_unlink+0x24d/0x7f0 [lustre] [ 5456.263790] vfs_unlink+0x202/0x330 [ 5456.265517] do_unlinkat+0x2fa/0x460 [ 5456.269467] __x64_sys_unlink+0x37/0x50 [ 5456.270508] do_syscall_64+0xc1/0x440 [ 5456.271937] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5456.276066] RIP: 0033:0x7f33957fc10b [ 5456.279901] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5456.294596] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5456.300051] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5456.303746] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5456.314211] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5456.317927] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5456.326919] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5460.058017] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 5460.063538] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5460.092992] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5460.097174] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5460.099920] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5460.103056] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5460.105254] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5460.111984] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5460.114628] RAX: 0000000000003a88 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5460.116666] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5460.120491] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5460.124148] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5460.127577] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5460.136996] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5460.140761] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5460.143565] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5460.150955] Call Trace: [ 5460.152133] [ 5460.153156] ? show_regs.cold.9+0x22/0x2f [ 5460.155029] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5460.157467] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5460.159652] ? watchdog+0x40/0x40 [ 5460.161194] ? hrtimer_interrupt+0x128/0x2c0 [ 5460.162680] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5460.165286] ? apic_timer_interrupt+0xf/0x20 [ 5460.170738] [ 5460.173673] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5460.176822] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5460.184063] do_raw_spin_lock+0x177/0x190 [ 5460.187516] _raw_spin_lock+0x21/0x30 [ 5460.189516] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5460.192693] process_one_work+0x2c8/0x700 [ 5460.194953] worker_thread+0x296/0x6e0 [ 5460.196726] ? rescuer_thread+0x570/0x570 [ 5460.199176] kthread+0x1d1/0x200 [ 5460.201125] ? set_kthread_struct+0x70/0x70 [ 5460.203426] ret_from_fork+0x1f/0x30 [ 5480.054320] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5480.064316] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5480.085354] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5480.090567] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5480.094481] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5480.098054] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5480.106138] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5480.109140] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5480.112518] RDX: 0000000000006974 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5480.117005] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5480.120295] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5480.124748] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5480.127381] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5480.130564] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5480.133751] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5480.137399] Call Trace: [ 5480.138444] [ 5480.140103] ? show_regs.cold.9+0x22/0x2f [ 5480.142199] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5480.144394] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5480.149263] ? watchdog+0x40/0x40 [ 5480.150866] ? hrtimer_interrupt+0x128/0x2c0 [ 5480.152713] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5480.155623] ? apic_timer_interrupt+0xf/0x20 [ 5480.157759] [ 5480.158521] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5480.161086] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5480.163353] do_raw_spin_lock+0x177/0x190 [ 5480.164747] _raw_spin_lock+0x21/0x30 [ 5480.167202] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5480.171378] obd_mod_free+0x46/0x140 [ptlrpc] [ 5480.174815] mdc_commit_open+0x91/0xb0 [mdc] [ 5480.177325] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5480.180011] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5480.183859] after_reply+0xe2c/0x1780 [ptlrpc] [ 5480.186030] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5480.192513] ? __next_timer_interrupt+0x160/0x160 [ 5480.194482] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5480.196398] ? do_wait_intr+0xf0/0xf0 [ 5480.197712] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5480.202642] kthread+0x1d1/0x200 [ 5480.203688] ? set_kthread_struct+0x70/0x70 [ 5480.207682] ret_from_fork+0x1f/0x30 [ 5484.063969] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5484.072172] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5484.108336] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5484.120067] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5484.135310] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5484.144013] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5484.163251] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5484.169500] RAX: 000000000000018e RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5484.175193] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5484.183037] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5484.189336] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5484.196207] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5484.202525] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5484.207539] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5484.214161] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5484.219309] Call Trace: [ 5484.222166] [ 5484.225276] ? show_regs.cold.9+0x22/0x2f [ 5484.234920] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5484.246060] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5484.248901] ? watchdog+0x40/0x40 [ 5484.255918] ? hrtimer_interrupt+0x128/0x2c0 [ 5484.262092] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5484.269440] ? apic_timer_interrupt+0xf/0x20 [ 5484.274359] [ 5484.275147] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5484.278407] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5484.288751] do_raw_spin_lock+0x177/0x190 [ 5484.290156] _raw_spin_lock+0x21/0x30 [ 5484.300748] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5484.310491] ? do_raw_spin_unlock+0x75/0x190 [ 5484.314977] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5484.321174] ? do_wait_intr+0xf0/0xf0 [ 5484.326419] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5484.333048] mdc_reint+0x2d/0x110 [mdc] [ 5484.337877] mdc_unlink+0x658/0x870 [mdc] [ 5484.342868] lmv_unlink+0x34d/0xbd0 [lmv] [ 5484.344939] ll_unlink+0x24d/0x7f0 [lustre] [ 5484.347959] vfs_unlink+0x202/0x330 [ 5484.349226] do_unlinkat+0x2fa/0x460 [ 5484.352112] __x64_sys_unlink+0x37/0x50 [ 5484.354153] do_syscall_64+0xc1/0x440 [ 5484.355684] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5484.357467] RIP: 0033:0x7f33957fc10b [ 5484.360906] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5484.385352] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5484.405449] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5484.415951] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5484.423349] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5484.427002] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5484.434323] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5488.058444] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 5488.065482] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5488.115824] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5488.124177] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5488.128297] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5488.131735] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5488.134290] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5488.145017] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5488.147416] RAX: 0000000000005907 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5488.152253] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5488.160022] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5488.166111] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5488.171469] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5488.175624] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5488.182762] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5488.185676] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5488.189022] Call Trace: [ 5488.193358] [ 5488.194450] ? show_regs.cold.9+0x22/0x2f [ 5488.196155] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5488.198589] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5488.200720] ? watchdog+0x40/0x40 [ 5488.202418] ? hrtimer_interrupt+0x128/0x2c0 [ 5488.204120] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5488.206184] ? apic_timer_interrupt+0xf/0x20 [ 5488.207736] [ 5488.208423] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5488.211121] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5488.214422] do_raw_spin_lock+0x177/0x190 [ 5488.218117] _raw_spin_lock+0x21/0x30 [ 5488.223453] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5488.229594] process_one_work+0x2c8/0x700 [ 5488.232209] worker_thread+0x296/0x6e0 [ 5488.234076] ? rescuer_thread+0x570/0x570 [ 5488.236097] kthread+0x1d1/0x200 [ 5488.237966] ? set_kthread_struct+0x70/0x70 [ 5488.239699] ret_from_fork+0x1f/0x30 [ 5508.053016] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5508.060980] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5508.093612] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5508.104217] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5508.116971] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5508.122975] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5508.151272] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5508.161485] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5508.171308] RDX: 0000000000006639 RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5508.177971] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5508.187502] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5508.198319] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5508.203846] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5508.208661] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5508.221685] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5508.237102] Call Trace: [ 5508.238469] [ 5508.239852] ? show_regs.cold.9+0x22/0x2f [ 5508.242763] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5508.250033] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5508.255968] ? watchdog+0x40/0x40 [ 5508.257488] ? hrtimer_interrupt+0x128/0x2c0 [ 5508.264863] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5508.271897] ? apic_timer_interrupt+0xf/0x20 [ 5508.275023] [ 5508.276470] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5508.280295] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5508.282998] do_raw_spin_lock+0x177/0x190 [ 5508.284963] _raw_spin_lock+0x21/0x30 [ 5508.286523] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5508.290415] obd_mod_free+0x46/0x140 [ptlrpc] [ 5508.293908] mdc_commit_open+0x91/0xb0 [mdc] [ 5508.296485] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5508.300110] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5508.303369] after_reply+0xe2c/0x1780 [ptlrpc] [ 5508.306323] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5508.308662] ? __next_timer_interrupt+0x160/0x160 [ 5508.312505] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5508.317590] ? do_wait_intr+0xf0/0xf0 [ 5508.320510] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5508.324671] kthread+0x1d1/0x200 [ 5508.326852] ? set_kthread_struct+0x70/0x70 [ 5508.328868] ret_from_fork+0x1f/0x30 [ 5512.059597] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5512.069025] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5512.105743] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5512.114685] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5512.127955] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5512.129809] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5512.149933] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5512.158204] RAX: 0000000000001878 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5512.167811] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5512.176321] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5512.184928] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5512.196441] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5512.205388] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5512.212969] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5512.218835] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5512.227152] Call Trace: [ 5512.229026] [ 5512.230874] ? show_regs.cold.9+0x22/0x2f [ 5512.233690] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5512.236407] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5512.240356] ? watchdog+0x40/0x40 [ 5512.243649] ? hrtimer_interrupt+0x128/0x2c0 [ 5512.246396] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5512.250016] ? apic_timer_interrupt+0xf/0x20 [ 5512.252954] [ 5512.253886] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5512.255564] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5512.258881] do_raw_spin_lock+0x177/0x190 [ 5512.260908] _raw_spin_lock+0x21/0x30 [ 5512.265810] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5512.274838] ? do_raw_spin_unlock+0x75/0x190 [ 5512.280372] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5512.287231] ? do_wait_intr+0xf0/0xf0 [ 5512.291634] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5512.301441] mdc_reint+0x2d/0x110 [mdc] [ 5512.306477] mdc_unlink+0x658/0x870 [mdc] [ 5512.308782] lmv_unlink+0x34d/0xbd0 [lmv] [ 5512.314742] ll_unlink+0x24d/0x7f0 [lustre] [ 5512.318625] vfs_unlink+0x202/0x330 [ 5512.320683] do_unlinkat+0x2fa/0x460 [ 5512.322744] __x64_sys_unlink+0x37/0x50 [ 5512.325214] do_syscall_64+0xc1/0x440 [ 5512.326473] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5512.329281] RIP: 0033:0x7f33957fc10b [ 5512.330906] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5512.341394] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5512.345785] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5512.349347] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5512.354281] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5512.358826] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5512.363616] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5516.060478] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 5516.064945] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5516.120300] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5516.132481] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5516.139078] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5516.144945] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5516.149702] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5516.164741] RSP: 0018:ffff985380da3dd8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5516.174214] RAX: 0000000000007e96 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5516.179964] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5516.183703] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5516.189566] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5516.192181] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5516.201905] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5516.206456] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5516.213051] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5516.215701] Call Trace: [ 5516.216925] [ 5516.217592] ? show_regs.cold.9+0x22/0x2f [ 5516.219571] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5516.225848] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5516.230777] ? watchdog+0x40/0x40 [ 5516.232041] ? hrtimer_interrupt+0x128/0x2c0 [ 5516.235538] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5516.240658] ? apic_timer_interrupt+0xf/0x20 [ 5516.242154] [ 5516.242921] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5516.246327] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5516.251643] do_raw_spin_lock+0x177/0x190 [ 5516.253164] _raw_spin_lock+0x21/0x30 [ 5516.256092] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5516.259066] process_one_work+0x2c8/0x700 [ 5516.265259] worker_thread+0x296/0x6e0 [ 5516.268305] ? rescuer_thread+0x570/0x570 [ 5516.269955] kthread+0x1d1/0x200 [ 5516.273688] ? set_kthread_struct+0x70/0x70 [ 5516.276632] ret_from_fork+0x1f/0x30 [ 5536.054282] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [ptlrpcd_rcv:2387] [ 5536.066800] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5536.139472] CPU: 0 PID: 2387 Comm: ptlrpcd_rcv Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5536.145831] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5536.152003] RIP: 0010:__pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5536.155298] Code: 0c 00 00 00 00 41 be 01 00 00 00 41 bf 00 01 00 00 65 48 ff 05 ed 8a c7 72 c6 45 14 00 ba 00 80 00 00 c6 43 01 01 eb 13 f3 90 <48> 83 05 5c d6 dc 02 01 83 ea 01 0f 84 ef 03 00 00 0f b6 03 84 c0 [ 5536.165901] RSP: 0018:ffff9853816cfbc8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 [ 5536.170026] RAX: 0000000000000003 RBX: ffff898e84d1d2e0 RCX: ffff898ec201e480 [ 5536.174485] RDX: 000000000000390c RSI: 0000000000000003 RDI: ffff898e84d1d2e0 [ 5536.187878] RBP: ffff898ec2034140 R08: 0000000000000008 R09: 0000000000000008 [ 5536.202070] R10: ffffffff8ee08b80 R11: ffff9853816cf8f0 R12: 0000000000040000 [ 5536.218505] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000100 [ 5536.229904] FS: 0000000000000000(0000) GS:ffff898ec2000000(0000) knlGS:0000000000000000 [ 5536.250249] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5536.268530] CR2: 00007f98f2861270 CR3: 000000010aa16001 CR4: 0000000000170ef0 [ 5536.290196] Call Trace: [ 5536.291041] [ 5536.312077] ? show_regs.cold.9+0x22/0x2f [ 5536.313459] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5536.323493] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5536.332015] ? watchdog+0x40/0x40 [ 5536.334910] ? hrtimer_interrupt+0x128/0x2c0 [ 5536.336554] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5536.343496] ? apic_timer_interrupt+0xf/0x20 [ 5536.350240] [ 5536.356287] ? __pv_queued_spin_lock_slowpath+0x174/0x6d0 [ 5536.368067] ? __pv_queued_spin_lock_slowpath+0x5e7/0x6d0 [ 5536.376844] do_raw_spin_lock+0x177/0x190 [ 5536.378323] _raw_spin_lock+0x21/0x30 [ 5536.385429] __ptlrpc_req_put+0x6af/0xa50 [ptlrpc] [ 5536.390421] obd_mod_free+0x46/0x140 [ptlrpc] [ 5536.392185] mdc_commit_open+0x91/0xb0 [mdc] [ 5536.397349] ptlrpc_free_request+0x4b/0xc0 [ptlrpc] [ 5536.401956] ptlrpc_free_committed+0x15c/0xd70 [ptlrpc] [ 5536.409407] after_reply+0xe2c/0x1780 [ptlrpc] [ 5536.413845] ptlrpc_check_set+0x2575/0x30b0 [ptlrpc] [ 5536.424858] ? __next_timer_interrupt+0x160/0x160 [ 5536.435706] ptlrpcd+0xa82/0xe40 [ptlrpc] [ 5536.437508] ? do_wait_intr+0xf0/0xf0 [ 5536.447346] ? ptlrpcd_add_req+0x540/0x540 [ptlrpc] [ 5536.450872] kthread+0x1d1/0x200 [ 5536.453267] ? set_kthread_struct+0x70/0x70 [ 5536.456211] ret_from_fork+0x1f/0x30 [ 5540.059338] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [writemany:37219] [ 5540.076608] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5540.149640] CPU: 3 PID: 37219 Comm: writemany Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5540.171967] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5540.179689] RIP: 0010:__pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5540.192939] Code: 4c 03 2c c5 80 58 c8 8e 49 89 6d 00 b8 00 80 00 00 eb 20 41 0f b6 55 14 48 83 05 46 d4 dc 02 01 84 d2 0f 85 db 00 00 00 f3 90 <83> e8 01 0f 84 3b 02 00 00 8b 55 08 85 d2 75 0e 84 c0 74 d5 48 83 [ 5540.213886] RSP: 0018:ffff985380d7fba8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 [ 5540.226824] RAX: 0000000000007b48 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5540.237886] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff898ec21b4154 [ 5540.248885] RBP: ffff898ec21b4140 R08: 0000000000000000 R09: 0000000000000000 [ 5540.262111] R10: 000000002e49b6d3 R11: ffff898e84e5e000 R12: 0000000000100000 [ 5540.272851] R13: ffff898ec2034140 R14: 000000000001e480 R15: ffff898ec219e480 [ 5540.282938] FS: 00007f3395fc2740(0000) GS:ffff898ec2180000(0000) knlGS:0000000000000000 [ 5540.288750] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5540.296405] CR2: 00007f34d7aab160 CR3: 00000001063d6003 CR4: 0000000000170ee0 [ 5540.300883] Call Trace: [ 5540.302532] [ 5540.303550] ? show_regs.cold.9+0x22/0x2f [ 5540.307098] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5540.314440] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5540.316963] ? watchdog+0x40/0x40 [ 5540.318685] ? hrtimer_interrupt+0x128/0x2c0 [ 5540.328068] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5540.337172] ? apic_timer_interrupt+0xf/0x20 [ 5540.349145] [ 5540.353947] ? __pv_queued_spin_lock_slowpath+0x30c/0x6d0 [ 5540.358968] ? __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5540.371798] do_raw_spin_lock+0x177/0x190 [ 5540.379545] _raw_spin_lock+0x21/0x30 [ 5540.383044] ptlrpc_check_set+0x11c5/0x30b0 [ptlrpc] [ 5540.392001] ? do_raw_spin_unlock+0x75/0x190 [ 5540.397842] ptlrpc_set_wait+0x38e/0xa50 [ptlrpc] [ 5540.411821] ? do_wait_intr+0xf0/0xf0 [ 5540.417140] ptlrpc_queue_wait+0xb1/0x2d0 [ptlrpc] [ 5540.428097] mdc_reint+0x2d/0x110 [mdc] [ 5540.442703] mdc_unlink+0x658/0x870 [mdc] [ 5540.455451] lmv_unlink+0x34d/0xbd0 [lmv] [ 5540.456929] ll_unlink+0x24d/0x7f0 [lustre] [ 5540.470644] vfs_unlink+0x202/0x330 [ 5540.481143] do_unlinkat+0x2fa/0x460 [ 5540.487349] __x64_sys_unlink+0x37/0x50 [ 5540.491094] do_syscall_64+0xc1/0x440 [ 5540.496312] entry_SYSCALL_64_after_hwframe+0x49/0xae [ 5540.512891] RIP: 0033:0x7f33957fc10b [ 5540.522165] Code: f0 ff ff 73 01 c3 48 8b 0d 7a 7d 39 00 f7 d8 64 89 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 57 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 7d 39 00 f7 d8 64 89 01 48 [ 5540.552920] RSP: 002b:00007ffe42a47a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000057 [ 5540.571769] RAX: ffffffffffffffda RBX: 000000000001df10 RCX: 00007f33957fc10b [ 5540.584759] RDX: 0000000000000038 RSI: 00007ffe42a47ee0 RDI: 00007ffe42a47ae0 [ 5540.600186] RBP: 00007ffe42a47ee0 R08: 00007f3395b9421c R09: 00007f3395b94260 [ 5540.611686] R10: 00007ffe42a47822 R11: 0000000000000246 R12: 0000000000000003 [ 5540.616257] R13: 00000000ffffffff R14: 0000000000000038 R15: 0000000000000000 [ 5544.058241] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u8:2:37] [ 5544.063127] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) ec(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl i2c_piix4 pcspkr squashfs ata_generic crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 5544.098108] CPU: 2 PID: 37 Comm: kworker/u8:2 Kdump: loaded Tainted: G O L -------- - - 4.18.0rh8.10-debug #2 [ 5544.108657] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5544.120074] Workqueue: ptlrpc_pinger ptlrpc_pinger_main [ptlrpc] [ 5544.127233] RIP: 0010:native_safe_halt+0x2e/0x50 [ 5544.131033] Code: ed f1 01 01 e9 2a 00 00 00 48 83 05 3b ed f1 01 01 48 83 05 3b ed f1 01 01 0f 00 2d 44 44 43 00 48 83 05 34 ed f1 01 01 fb f4 <48> 83 05 32 ed f1 01 01 c3 48 83 05 09 ed f1 01 01 eb eb 66 66 2e [ 5544.157820] RSP: 0018:ffff985380da3dc8 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13 [ 5544.168617] RAX: 0000000000000001 RBX: ffff898e84d1d2e0 RCX: 0000000000000001 [ 5544.176059] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff898ec2134154 [ 5544.187844] RBP: ffff898ec2134140 R08: 0000000000000000 R09: 0000000000000000 [ 5544.196799] R10: 000000002e49bce4 R11: ffff898e84f5d000 R12: 00000000000c0000 [ 5544.202383] R13: ffff898ec21b4140 R14: 000000000001e480 R15: ffff898ec211e480 [ 5544.211777] FS: 0000000000000000(0000) GS:ffff898ec2100000(0000) knlGS:0000000000000000 [ 5544.218142] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5544.227625] CR2: 0000563e44f60948 CR3: 000000010aa16005 CR4: 0000000000170ee0 [ 5544.236961] Call Trace: [ 5544.238724] [ 5544.240615] ? show_regs.cold.9+0x22/0x2f [ 5544.242680] ? watchdog_timer_fn.cold.6+0x62/0x10a [ 5544.255130] ? __hrtimer_run_queues+0x1c2/0x4d0 [ 5544.264360] ? watchdog+0x40/0x40 [ 5544.268229] ? hrtimer_interrupt+0x128/0x2c0 [ 5544.274797] ? smp_apic_timer_interrupt+0xa6/0x270 [ 5544.286020] ? apic_timer_interrupt+0xf/0x20 [ 5544.288130] [ 5544.291886] ? native_safe_halt+0x2e/0x50 [ 5544.298836] kvm_wait+0xcd/0xe0 [ 5544.300951] __pv_queued_spin_lock_slowpath+0x451/0x6d0 [ 5544.307751] do_raw_spin_lock+0x177/0x190 [ 5544.309796] _raw_spin_lock+0x21/0x30 [ 5544.312665] ptlrpc_pinger_main+0x1cd/0xf30 [ptlrpc] [ 5544.323256] process_one_work+0x2c8/0x700 [ 5544.325697] worker_thread+0x296/0x6e0 [ 5544.329095] ? rescuer_thread+0x570/0x570 [ 5544.334752] kthread+0x1d1/0x200 [ 5544.336533] ? set_kthread_struct+0x70/0x70 [ 5544.352805] ret_from_fork+0x1f/0x30