************************ crashinfo ************************* /exports/testreports/68568/testresults/sanity3-zfs-rocky8.10_x86_64-rocky8.10_x86_64-retry1/oleg329-client-vmcore (4.18.0rh8.10-debug) +==========================+ | *** Crashinfo v1.3.7 *** | +==========================+ +++WARNING+++ PARTIAL DUMP with size(vmcore) < 25% size(RAM) KERNEL: /tmp/crash-anaysis.EbSmX/vmlinux [TAINTED] DUMPFILE: /exports/testreports/68568/testresults/sanity3-zfs-rocky8.10_x86_64-rocky8.10_x86_64-retry1/oleg329-client-vmcore [PARTIAL DUMP] CPUS: 4 DATE: Mon Aug 24 02:30:59 EDT 2026 UPTIME: 01:21:48 LOAD AVERAGE: 1.71, 2.14, 1.76 TASKS: 197 NODENAME: oleg329-client.virtnet RELEASE: 4.18.0rh8.10-debug VERSION: #2 SMP Mon Jul 14 01:24:22 EDT 2025 MACHINE: x86_64 (2399 Mhz) MEMORY: 4.1 GB PANIC: "BUG: unable to handle kernel paging request at 00000003cdbfdd80" +--------------------------+ >------------------------| Per-cpu Stacks ('bt -a') |------------------------< +--------------------------+ -- CPU#0 -- PID=144538 CPU=0 CMD=fio #0 crash_nmi_callback+0x6d #1 nmi_handle+0xbd #2 default_do_nmi+0x4b #3 do_nmi+0x280 #4 end_repeat_nmi+0x16 #-1 native_safe_halt+0x2e, 507 bytes of data #5 native_safe_halt+0x2e #6 kvm_wait+0xcd #7 __pv_queued_spin_lock_slowpath+0x5e7 #8 do_raw_spin_lock+0x177 #9 _raw_spin_lock+0x21 #10 osc_enter_cache+0x6e8 #11 osc_queue_async_io+0x604 #12 osc_page_cache_add+0x66 #13 osc_io_commit_async+0x2b0 #14 cl_io_commit_async+0x174 #15 lov_io_commit_async+0x3f3 #16 cl_io_commit_async+0x174 #17 vvp_io_write_commit+0x1cb #18 vvp_io_write_start+0x9a6 #19 cl_io_start+0x65 #20 cl_io_loop+0xe2 #21 ll_file_io_generic+0x686 #22 do_file_write_iter+0x8f2 #23 ll_file_write_iter+0x3d #24 new_sync_write+0x160 #25 __vfs_write+0x39 #26 vfs_write+0xd8 #27 ksys_pwrite64+0x81 #28 __x64_sys_pwrite64+0x22 #29 do_syscall_64+0xc1 #30 entry_SYSCALL_64_after_hwframe+0x49, 477 bytes of data -- CPU#1 -- PID=0 CPU=1 CMD=swapper/1 #0 crash_nmi_callback+0x6d #1 nmi_handle+0xbd #2 default_do_nmi+0x4b #3 do_nmi+0x280 #4 end_repeat_nmi+0x16 #-1 native_safe_halt+0x2e, 507 bytes of data #5 native_safe_halt+0x2e #6 default_idle+0x1a #7 arch_cpu_idle+0x13 #8 default_idle_call+0xb8 #9 do_idle+0x3c9 #10 cpu_startup_entry+0x9f #11 start_secondary+0x2b8 #12 secondary_startup_64_no_verify+0xd1 -- CPU#2 -- PID=134199 CPU=2 CMD=kworker/2:3 #0 crash_nmi_callback+0x6d #1 nmi_handle+0xbd #2 default_do_nmi+0x4b #3 do_nmi+0x280 #4 end_repeat_nmi+0x16 #-1 __pv_queued_spin_lock_slowpath+0x30c, 507 bytes of data #5 __pv_queued_spin_lock_slowpath+0x30c #6 do_raw_spin_lock+0x177 #7 _raw_spin_lock+0x21 #8 osc_announce_cached+0x3d #9 osc_brw_prep_request+0x23df #10 osc_build_rpc+0x1207 #11 osc_check_rpcs+0x124b #12 __osc_io_unplug+0x11b #13 brw_queue_work+0x66 #14 process_one_work+0x2c8 #15 worker_thread+0x296 #16 kthread+0x1d1 #17 ret_from_fork+0x1f -- CPU#3 -- PID=144535 CPU=3 CMD=fio #0 machine_kexec+0x243 #1 __crash_kexec+0xaa #2 crash_kexec+0x65 #3 oops_end+0x155 #4 no_context.cold.27+0xa6 #5 __bad_area_nosemaphore+0x1c6 #6 bad_area_nosemaphore+0x1a #7 do_user_addr_fault+0x540 #8 __do_page_fault+0x6b #9 do_page_fault+0x87 #10 page_fault+0x1e #-1 __osc_extent_tree_dump+0x403, 477 bytes of data #11 osc_enter_cache+0x873 #12 osc_queue_async_io+0x604 #13 osc_page_cache_add+0x66 #14 osc_io_commit_async+0x2b0 #15 cl_io_commit_async+0x174 #16 lov_io_commit_async+0x3f3 #17 cl_io_commit_async+0x174 #18 vvp_io_write_commit+0x1cb #19 vvp_io_write_start+0x9a6 #20 cl_io_start+0x65 #21 cl_io_loop+0xe2 #22 ll_file_io_generic+0x686 #23 do_file_write_iter+0x8f2 #24 ll_file_write_iter+0x3d #25 new_sync_write+0x160 #26 __vfs_write+0x39 #27 vfs_write+0xd8 #28 ksys_pwrite64+0x81 #29 __x64_sys_pwrite64+0x22 #30 do_syscall_64+0xc1 #31 entry_SYSCALL_64_after_hwframe+0x49, 477 bytes of data +--------------------------------+ >---------------------| How This Dump Has Been Created |---------------------< +--------------------------------+ *** Dump was triggered by kernel *** BUG: unable to handle kernel paging request at 00000003cdbfdd80 +---------------+ >------------------------------| Tasks Summary |------------------------------< +---------------+ +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task +++ERROR+++ corrupted task Number of Threads That Ran Recently ----------------------------------- last second 23 last 5s 42 last 60s 73 ----- Total Numbers of Threads per State ------ ?? 194 +++WARNING+++ There are 6 threads running in their own namespaces Use 'taskinfo --ns' to get more details +-----------------------+ >--------------------------| 5 Most Recent Threads |--------------------------< +-----------------------+ PID CMD Age ARGS ----- -------------- ------ ---------------------------- 144535 fio 0 ms (no user stack) 1365 kworker/u8:4 0 ms (no user stack) 144526 fio 7 ms (no user stack) 144527 fio 7 ms (no user stack) 1351 tuned 7 ms (no user stack) +------------------------+ >-------------------------| Memory Usage (kmem -i) |-------------------------< +------------------------+ PAGES TOTAL PERCENTAGE TOTAL MEM 945455 3.6 GB ---- FREE 218672 854.2 MB 23% of TOTAL MEM USED 726783 2.8 GB 76% of TOTAL MEM SHARED 589464 2.2 GB 62% of TOTAL MEM BUFFERS 3638 14.2 MB 0% of TOTAL MEM CACHED 612301 2.3 GB 64% of TOTAL MEM SLAB 9328 36.4 MB 0% of TOTAL MEM TOTAL HUGE 0 0 ---- HUGE FREE 0 0 0% of TOTAL HUGE TOTAL SWAP 262143 1024 MB ---- SWAP USED 0 0 0% of TOTAL SWAP SWAP FREE 262143 1024 MB 100% of TOTAL SWAP COMMIT LIMIT 734870 2.8 GB ---- COMMITTED 1097698 4.2 GB 149% of TOTAL LIMIT print_blkreq is not supported on this kernel Traceback (most recent call last): File "/usr/local/crash/mpykdump.so/progs/crashinfo.py", line 1673, in File "/home/green/git/pykdump/LinuxDump/Dev.py", line 500, in print_gendisk # Get the list of (dev, gendisk) sorted by dev File "/home/green/git/pykdump/LinuxDump/Dev.py", line 473, in get_gendisks ttype = target.type File "/home/green/git/pykdump/pykdump/lowlevel.py", line 391, in __getattr__ raise KeyError(msg) KeyError: ' does not have a field ' ****************************************************************************** ************************ A Summary Of Problems Found ************************* ****************************************************************************** -------------------- A list of all +++WARNING+++ messages -------------------- PARTIAL DUMP with size(vmcore) < 25% size(RAM) There are 6 threads running in their own namespaces Use 'taskinfo --ns' to get more details --------------------- A list of all +++ERROR+++ messages --------------------- corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task corrupted task ------------------------------------------------------------------------------ ** Execution took 22.62s (real) 8.29s (CPU), Child processes: 13.91s