crash 8.0.0++ Copyright (C) 2002-2021 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011, 2020-2021 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. Copyright (C) 2015, 2021 VMware, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. NOTE: stdin: not a tty GNU gdb (GDB) 10.2 Copyright (C) 2021 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... please wait... (gathering kmem slab cache data) please wait... (gathering module symbol data) please wait... (gathering task table data) please wait... (determining panic task) KERNEL: /tmp/crash-anaysis.6m3B6/vmlinux [TAINTED] DUMPFILE: /exports/testreports/42545/testresults/replay-single-zfs-centos7_x86_64-centos7_x86_64/oleg408-server-vmcore [PARTIAL DUMP] CPUS: 4 DATE: Wed May 8 18:02:57 EDT 2024 UPTIME: 00:50:37 LOAD AVERAGE: 0.68, 0.38, 0.35 TASKS: 459 NODENAME: oleg408-server.virtnet RELEASE: 3.10.0-7.9-debug VERSION: #1 SMP Sat Mar 26 23:28:42 EDT 2022 MACHINE: x86_64 (2399 Mhz) MEMORY: 4 GB PANIC: "Kernel panic - not syncing: LBUG" PID: 17642 COMMAND: "umount" TASK: ffff8800a6692220 [THREAD_INFO: ffff88009f480000] CPU: 2 STATE: TASK_RUNNING (PANIC) Setting scroll off while initializing PyKdump /usr/local/crash/mpykdump.so: shared object loaded ./lustre.so: shared object loaded mod: cannot find or load object file for libata module mod: cannot find or load object file for serio_raw module mod: cannot find or load object file for crc32c_intel module mod: cannot find or load object file for pata_acpi module mod: cannot find or load object file for ata_piix module mod: cannot find or load object file for ata_generic module mod: cannot find or load object file for crct10dif_pclmul module mod: cannot find or load object file for ip_tables module mod: cannot find or load object file for cryptd module mod: cannot find or load object file for crct10dif_common module mod: cannot find or load object file for jbd2 module mod: cannot find or load object file for mbcache module mod: cannot find or load object file for ext4 module mod: cannot find or load object file for binfmt_misc module mod: cannot find or load object file for ablk_helper module mod: cannot find or load object file for pcspkr module mod: cannot find or load object file for gf128mul module mod: cannot find or load object file for glue_helper module mod: cannot find or load object file for crct10dif_generic module mod: cannot find or load object file for irqbypass module mod: cannot find or load object file for i2c_core module mod: cannot find or load object file for lrw module mod: cannot find or load object file for i2c_piix4 module mod: cannot find or load object file for ghash_clmulni_intel module mod: cannot find or load object file for aesni_intel module mod: cannot find or load object file for crc_t10dif module mod: cannot find or load object file for crc32_pclmul module mod: cannot find or load object file for zavl module mod: cannot find or load object file for kvm module mod: cannot find or load object file for squashfs module mod: cannot find or load object file for rpcsec_gss_krb5 module MODULE NAME BASE SIZE OBJECT FILE ffffffffa02875c0 mgc ffffffffa0276000 99216 /tmp/crash-anaysis.6m3B6/modules/mgc.ko ffffffffa0292280 ost ffffffffa0290000 14932 /tmp/crash-anaysis.6m3B6/modules/ost.ko mod: cannot find or load object file for edac_core module mod: cannot find or load object file for sb_edac module ffffffffa02c2a20 fld ffffffffa02b4000 88993 /tmp/crash-anaysis.6m3B6/modules/fld.ko mod: cannot find or load object file for iosf_mbi module ffffffffa0307a20 libcfs ffffffffa02e3000 229581 /tmp/crash-anaysis.6m3B6/modules/libcfs.ko ffffffffa0384b20 lnet ffffffffa031d000 691552 /tmp/crash-anaysis.6m3B6/modules/lnet.ko ffffffffa03e4940 ksocklnd ffffffffa03c7000 178330 /tmp/crash-anaysis.6m3B6/modules/ksocklnd.ko ffffffffa042a4c0 ptlrpc_gss ffffffffa03f4000 336560 /tmp/crash-anaysis.6m3B6/modules/ptlrpc_gss.ko ffffffffa0456320 fid ffffffffa0448000 84965 /tmp/crash-anaysis.6m3B6/modules/fid.ko ffffffffa0482e80 lmv ffffffffa045e000 201459 /tmp/crash-anaysis.6m3B6/modules/lmv.ko ffffffffa04dbf40 osc ffffffffa0491000 422987 /tmp/crash-anaysis.6m3B6/modules/osc.ko mod: cannot find or load object file for kvm_intel module ffffffffa0606f40 obdclass ffffffffa0536000 2746832 /tmp/crash-anaysis.6m3B6/modules/obdclass.ko ffffffffa0968fc0 ptlrpc ffffffffa07d6000 2381154 /tmp/crash-anaysis.6m3B6/modules/ptlrpc.ko mod: cannot find or load object file for znvpair module ffffffffa0a71800 lov ffffffffa0a38000 328021 /tmp/crash-anaysis.6m3B6/modules/lov.ko ffffffffa0aba2a0 mdc ffffffffa0a8a000 266008 /tmp/crash-anaysis.6m3B6/modules/mdc.ko ffffffffa0ae8a40 obdecho ffffffffa0acc000 167686 /tmp/crash-anaysis.6m3B6/modules/obdecho.ko ffffffffa0b8d7c0 lfsck ffffffffa0af6000 784296 /tmp/crash-anaysis.6m3B6/modules/lfsck.ko mod: cannot find or load object file for spl module ffffffffa0c2c2e0 lquota ffffffffa0bd9000 458050 /tmp/crash-anaysis.6m3B6/modules/lquota.ko mod: cannot find or load object file for icp module mod: cannot find or load object file for zcommon module mod: cannot find or load object file for zlua module mod: cannot find or load object file for zunicode module mod: cannot find or load object file for zfs module ffffffffa121a1c0 mgs ffffffffa11d7000 369001 /tmp/crash-anaysis.6m3B6/modules/mgs.ko ffffffffa1292d00 osd_zfs ffffffffa124b000 395722 /tmp/crash-anaysis.6m3B6/modules/osd_zfs.ko ffffffffa13079a0 mdd ffffffffa12ad000 487413 /tmp/crash-anaysis.6m3B6/modules/mdd.ko ffffffffa13f0d40 mdt ffffffffa133e000 987350 /tmp/crash-anaysis.6m3B6/modules/mdt.ko ffffffffa14a1540 lod ffffffffa1431000 611564 /tmp/crash-anaysis.6m3B6/modules/lod.ko ffffffffa1526680 osp ffffffffa14e8000 350863 /tmp/crash-anaysis.6m3B6/modules/osp.ko ffffffffa1580480 ofd ffffffffa153f000 357053 /tmp/crash-anaysis.6m3B6/modules/ofd.ko ffffffffa164f240 lustre ffffffffa1598000 1002973 /tmp/crash-anaysis.6m3B6/modules/lustre.ko lustre_walk_cpus(0, 5, 1) cmd p (*cfs_trace_data[0])[0].tcd.tcd_cur_pages // p (*cfs_trace_data[0])[0].tcd.tcd_pages.next lustre_walk_trace_pages(0, 5, 0xffff88008f3e51e8) lustre_walk_cpus(0, 5, 2) cmd p (*cfs_trace_data[0])[0].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[0])[0].tcd.tcd_daemon_pages.next lustre: Dumped 409 debug pages from type 0 - CPU 0 lustre_walk_cpus(1, 5, 1) cmd p (*cfs_trace_data[0])[1].tcd.tcd_cur_pages // p (*cfs_trace_data[0])[1].tcd.tcd_pages.next lustre_walk_trace_pages(1, 5, 0xffff8800b1dddda8) lustre_walk_cpus(1, 5, 2) cmd p (*cfs_trace_data[0])[1].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[0])[1].tcd.tcd_daemon_pages.next lustre: Dumped 409 debug pages from type 0 - CPU 1 lustre_walk_cpus(2, 5, 1) cmd p (*cfs_trace_data[0])[2].tcd.tcd_cur_pages // p (*cfs_trace_data[0])[2].tcd.tcd_pages.next lustre_walk_trace_pages(2, 5, 0xffff8800b1f1a168) lustre_walk_cpus(2, 5, 2) cmd p (*cfs_trace_data[0])[2].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[0])[2].tcd.tcd_daemon_pages.next lustre: Dumped 409 debug pages from type 0 - CPU 2 lustre_walk_cpus(3, 5, 1) cmd p (*cfs_trace_data[0])[3].tcd.tcd_cur_pages // p (*cfs_trace_data[0])[3].tcd.tcd_pages.next lustre_walk_trace_pages(3, 5, 0xffff8800a65a27e8) lustre_walk_cpus(3, 5, 2) cmd p (*cfs_trace_data[0])[3].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[0])[3].tcd.tcd_daemon_pages.next lustre: Dumped 409 debug pages from type 0 - CPU 3 lustre_walk_cpus(0, 5, 1) cmd p (*cfs_trace_data[1])[0].tcd.tcd_cur_pages // p (*cfs_trace_data[1])[0].tcd.tcd_pages.next lustre_walk_cpus(0, 5, 2) cmd p (*cfs_trace_data[1])[0].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[1])[0].tcd.tcd_daemon_pages.next lustre: Dumped 0 debug pages from type 1 - CPU 0 lustre_walk_cpus(1, 5, 1) cmd p (*cfs_trace_data[1])[1].tcd.tcd_cur_pages // p (*cfs_trace_data[1])[1].tcd.tcd_pages.next lustre_walk_trace_pages(1, 5, 0xffff8800ac827be8) lustre_walk_cpus(1, 5, 2) cmd p (*cfs_trace_data[1])[1].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[1])[1].tcd.tcd_daemon_pages.next lustre: Dumped 1 debug pages from type 1 - CPU 1 lustre_walk_cpus(2, 5, 1) cmd p (*cfs_trace_data[1])[2].tcd.tcd_cur_pages // p (*cfs_trace_data[1])[2].tcd.tcd_pages.next lustre_walk_trace_pages(2, 5, 0xffff8800b4a69308) lustre_walk_cpus(2, 5, 2) cmd p (*cfs_trace_data[1])[2].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[1])[2].tcd.tcd_daemon_pages.next lustre: Dumped 1 debug pages from type 1 - CPU 2 lustre_walk_cpus(3, 5, 1) cmd p (*cfs_trace_data[1])[3].tcd.tcd_cur_pages // p (*cfs_trace_data[1])[3].tcd.tcd_pages.next lustre_walk_cpus(3, 5, 2) cmd p (*cfs_trace_data[1])[3].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[1])[3].tcd.tcd_daemon_pages.next lustre: Dumped 0 debug pages from type 1 - CPU 3 lustre_walk_cpus(0, 5, 1) cmd p (*cfs_trace_data[2])[0].tcd.tcd_cur_pages // p (*cfs_trace_data[2])[0].tcd.tcd_pages.next lustre_walk_cpus(0, 5, 2) cmd p (*cfs_trace_data[2])[0].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[2])[0].tcd.tcd_daemon_pages.next lustre: Dumped 0 debug pages from type 2 - CPU 0 lustre_walk_cpus(1, 5, 1) cmd p (*cfs_trace_data[2])[1].tcd.tcd_cur_pages // p (*cfs_trace_data[2])[1].tcd.tcd_pages.next lustre_walk_trace_pages(1, 5, 0xffff8800b534ac08) lustre_walk_cpus(1, 5, 2) cmd p (*cfs_trace_data[2])[1].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[2])[1].tcd.tcd_daemon_pages.next lustre: Dumped 1 debug pages from type 2 - CPU 1 lustre_walk_cpus(2, 5, 1) cmd p (*cfs_trace_data[2])[2].tcd.tcd_cur_pages // p (*cfs_trace_data[2])[2].tcd.tcd_pages.next lustre_walk_cpus(2, 5, 2) cmd p (*cfs_trace_data[2])[2].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[2])[2].tcd.tcd_daemon_pages.next lustre: Dumped 0 debug pages from type 2 - CPU 2 lustre_walk_cpus(3, 5, 1) cmd p (*cfs_trace_data[2])[3].tcd.tcd_cur_pages // p (*cfs_trace_data[2])[3].tcd.tcd_pages.next lustre_walk_trace_pages(3, 5, 0xffff8800a33ee228) lustre_walk_cpus(3, 5, 2) cmd p (*cfs_trace_data[2])[3].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[2])[3].tcd.tcd_daemon_pages.next lustre: Dumped 1 debug pages from type 2 - CPU 3 lustre: Dumped 1640 total debug pages from 4 CPUs to /tmp/crash-anaysis.6m3B6/lustre.bin