crash 8.0.0++ Copyright (C) 2002-2021 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011, 2020-2021 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. Copyright (C) 2015, 2021 VMware, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. NOTE: stdin: not a tty GNU gdb (GDB) 10.2 Copyright (C) 2021 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... please wait... (gathering kmem slab cache data) please wait... (gathering module symbol data) please wait... (gathering task table data) please wait... (determining panic task) KERNEL: /tmp/crash-anaysis.OmalY/vmlinux [TAINTED] DUMPFILE: /exports/testreports/37274/testresults/sanity-quota-ldiskfs-DNE-centos7_x86_64-centos7_x86_64/oleg345-client-vmcore [PARTIAL DUMP] CPUS: 4 DATE: Tue Oct 31 09:52:46 EDT 2023 UPTIME: 01:04:56 LOAD AVERAGE: 1.41, 0.49, 0.36 TASKS: 155 NODENAME: oleg345-client.virtnet RELEASE: 3.10.0-7.9-debug VERSION: #1 SMP Sat Mar 26 23:28:42 EDT 2022 MACHINE: x86_64 (2399 Mhz) MEMORY: 4 GB PANIC: "Kernel panic - not syncing: softlockup: hung tasks" PID: 1855 COMMAND: "ptlrpcd_00_00" TASK: ffff88012b245550 [THREAD_INFO: ffff8800ae224000] CPU: 1 STATE: TASK_RUNNING (PANIC) Setting scroll off while initializing PyKdump /usr/local/crash/mpykdump.so: shared object loaded ./lustre.so: shared object loaded mod: cannot find or load object file for libata module mod: cannot find or load object file for pata_acpi module mod: cannot find or load object file for ata_generic module mod: cannot find or load object file for mbcache module mod: cannot find or load object file for pcspkr module mod: cannot find or load object file for serio_raw module mod: cannot find or load object file for ata_piix module mod: cannot find or load object file for jbd2 module mod: cannot find or load object file for ext4 module mod: cannot find or load object file for binfmt_misc module mod: cannot find or load object file for crct10dif_common module mod: cannot find or load object file for crc_t10dif module mod: cannot find or load object file for squashfs module mod: cannot find or load object file for ip_tables module mod: cannot find or load object file for rpcsec_gss_krb5 module mod: cannot find or load object file for i2c_core module mod: cannot find or load object file for i2c_piix4 module mod: cannot find or load object file for crct10dif_generic module MODULE NAME BASE SIZE OBJECT FILE ffffffffa016a280 ost ffffffffa0168000 14891 /tmp/crash-anaysis.OmalY/modules/ost.ko ffffffffa018cd60 libcfs ffffffffa016d000 202723 /tmp/crash-anaysis.OmalY/modules/libcfs.ko mod: cannot find or load object file for crc32_generic module ffffffffa0232a60 lnet ffffffffa01b8000 800371 /tmp/crash-anaysis.OmalY/modules/lnet.ko ffffffffa028a1a0 fid ffffffffa027d000 80213 /tmp/crash-anaysis.OmalY/modules/fid.ko ffffffffa02c3880 ksocklnd ffffffffa02a6000 178047 /tmp/crash-anaysis.OmalY/modules/ksocklnd.ko ffffffffa0399b40 obdclass ffffffffa02d3000 2616723 /tmp/crash-anaysis.OmalY/modules/obdclass.ko ffffffffa058c0a0 ptlrpc_gss ffffffffa0553000 350768 /tmp/crash-anaysis.OmalY/modules/ptlrpc_gss.ko ffffffffa075ab80 ptlrpc ffffffffa05aa000 2556269 /tmp/crash-anaysis.OmalY/modules/ptlrpc.ko ffffffffa082a7e0 fld ffffffffa081c000 88791 /tmp/crash-anaysis.OmalY/modules/fld.ko ffffffffa085b5c0 lmv ffffffffa0833000 220867 /tmp/crash-anaysis.OmalY/modules/lmv.ko ffffffffa08b4bc0 osc ffffffffa086a000 422828 /tmp/crash-anaysis.OmalY/modules/osc.ko ffffffffa0927460 lov ffffffffa08ee000 323427 /tmp/crash-anaysis.OmalY/modules/lov.ko ffffffffa096f4a0 mdc ffffffffa093e000 271348 /tmp/crash-anaysis.OmalY/modules/mdc.ko ffffffffa0994a60 mgc ffffffffa0982000 109511 /tmp/crash-anaysis.OmalY/modules/mgc.ko ffffffffa09b7120 obdecho ffffffffa099e000 151561 /tmp/crash-anaysis.OmalY/modules/obdecho.ko ffffffffa0a5c200 lfsck ffffffffa09c5000 782301 /tmp/crash-anaysis.OmalY/modules/lfsck.ko ffffffffa0afbfc0 lquota ffffffffa0aa7000 466710 /tmp/crash-anaysis.OmalY/modules/lquota.ko ffffffffa0b5e1a0 mgs ffffffffa0b1a000 373735 /tmp/crash-anaysis.OmalY/modules/mgs.ko ffffffffa0bd1d00 mdd ffffffffa0b77000 489426 /tmp/crash-anaysis.OmalY/modules/mdd.ko ffffffffa0ca6840 mdt ffffffffa0bf0000 1004763 /tmp/crash-anaysis.OmalY/modules/mdt.ko ffffffffa0d5e7c0 lod ffffffffa0ce7000 651466 /tmp/crash-anaysis.OmalY/modules/lod.ko ffffffffa0de8240 osp ffffffffa0daa000 346273 /tmp/crash-anaysis.OmalY/modules/osp.ko ffffffffa0e41360 ofd ffffffffa0e00000 356847 /tmp/crash-anaysis.OmalY/modules/ofd.ko ffffffffa0f15b00 lustre ffffffffa0e59000 1029071 /tmp/crash-anaysis.OmalY/modules/lustre.ko lustre_walk_cpus(0, 5, 1) cmd p (*cfs_trace_data[0])[0].tcd.tcd_cur_pages // p (*cfs_trace_data[0])[0].tcd.tcd_pages.next lustre_walk_trace_pages(0, 5, 0xffff8800b1cf6b28) lustre_walk_cpus(0, 5, 2) cmd p (*cfs_trace_data[0])[0].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[0])[0].tcd.tcd_daemon_pages.next lustre: Dumped 409 debug pages from type 0 - CPU 0 lustre_walk_cpus(1, 5, 1) cmd p (*cfs_trace_data[0])[1].tcd.tcd_cur_pages // p (*cfs_trace_data[0])[1].tcd.tcd_pages.next lustre_walk_trace_pages(1, 5, 0xffff8800ae0135a8) lustre_walk_cpus(1, 5, 2) cmd p (*cfs_trace_data[0])[1].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[0])[1].tcd.tcd_daemon_pages.next lustre: Dumped 409 debug pages from type 0 - CPU 1 lustre_walk_cpus(2, 5, 1) cmd p (*cfs_trace_data[0])[2].tcd.tcd_cur_pages // p (*cfs_trace_data[0])[2].tcd.tcd_pages.next lustre_walk_trace_pages(2, 5, 0xffff8800b1f204c8) lustre_walk_cpus(2, 5, 2) cmd p (*cfs_trace_data[0])[2].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[0])[2].tcd.tcd_daemon_pages.next lustre: Dumped 409 debug pages from type 0 - CPU 2 lustre_walk_cpus(3, 5, 1) cmd p (*cfs_trace_data[0])[3].tcd.tcd_cur_pages // p (*cfs_trace_data[0])[3].tcd.tcd_pages.next lustre_walk_trace_pages(3, 5, 0xffff8800ae014c28) lustre_walk_cpus(3, 5, 2) cmd p (*cfs_trace_data[0])[3].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[0])[3].tcd.tcd_daemon_pages.next lustre: Dumped 409 debug pages from type 0 - CPU 3 lustre_walk_cpus(0, 5, 1) cmd p (*cfs_trace_data[1])[0].tcd.tcd_cur_pages // p (*cfs_trace_data[1])[0].tcd.tcd_pages.next lustre_walk_trace_pages(0, 5, 0xffff8800abbb7508) lustre_walk_cpus(0, 5, 2) cmd p (*cfs_trace_data[1])[0].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[1])[0].tcd.tcd_daemon_pages.next lustre: Dumped 1 debug pages from type 1 - CPU 0 lustre_walk_cpus(1, 5, 1) cmd p (*cfs_trace_data[1])[1].tcd.tcd_cur_pages // p (*cfs_trace_data[1])[1].tcd.tcd_pages.next lustre_walk_cpus(1, 5, 2) cmd p (*cfs_trace_data[1])[1].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[1])[1].tcd.tcd_daemon_pages.next lustre: Dumped 0 debug pages from type 1 - CPU 1 lustre_walk_cpus(2, 5, 1) cmd p (*cfs_trace_data[1])[2].tcd.tcd_cur_pages // p (*cfs_trace_data[1])[2].tcd.tcd_pages.next lustre_walk_trace_pages(2, 5, 0xffff8800ae00a8a8) lustre_walk_cpus(2, 5, 2) cmd p (*cfs_trace_data[1])[2].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[1])[2].tcd.tcd_daemon_pages.next lustre: Dumped 1 debug pages from type 1 - CPU 2 lustre_walk_cpus(3, 5, 1) cmd p (*cfs_trace_data[1])[3].tcd.tcd_cur_pages // p (*cfs_trace_data[1])[3].tcd.tcd_pages.next lustre_walk_cpus(3, 5, 2) cmd p (*cfs_trace_data[1])[3].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[1])[3].tcd.tcd_daemon_pages.next lustre: Dumped 0 debug pages from type 1 - CPU 3 lustre_walk_cpus(0, 5, 1) cmd p (*cfs_trace_data[2])[0].tcd.tcd_cur_pages // p (*cfs_trace_data[2])[0].tcd.tcd_pages.next lustre_walk_cpus(0, 5, 2) cmd p (*cfs_trace_data[2])[0].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[2])[0].tcd.tcd_daemon_pages.next lustre: Dumped 0 debug pages from type 2 - CPU 0 lustre_walk_cpus(1, 5, 1) cmd p (*cfs_trace_data[2])[1].tcd.tcd_cur_pages // p (*cfs_trace_data[2])[1].tcd.tcd_pages.next lustre_walk_cpus(1, 5, 2) cmd p (*cfs_trace_data[2])[1].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[2])[1].tcd.tcd_daemon_pages.next lustre: Dumped 0 debug pages from type 2 - CPU 1 lustre_walk_cpus(2, 5, 1) cmd p (*cfs_trace_data[2])[2].tcd.tcd_cur_pages // p (*cfs_trace_data[2])[2].tcd.tcd_pages.next lustre_walk_cpus(2, 5, 2) cmd p (*cfs_trace_data[2])[2].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[2])[2].tcd.tcd_daemon_pages.next lustre: Dumped 0 debug pages from type 2 - CPU 2 lustre_walk_cpus(3, 5, 1) cmd p (*cfs_trace_data[2])[3].tcd.tcd_cur_pages // p (*cfs_trace_data[2])[3].tcd.tcd_pages.next lustre_walk_cpus(3, 5, 2) cmd p (*cfs_trace_data[2])[3].tcd.tcd_cur_daemon_pages // p (*cfs_trace_data[2])[3].tcd.tcd_daemon_pages.next lustre: Dumped 0 debug pages from type 2 - CPU 3 lustre: Dumped 1638 total debug pages from 4 CPUs to /tmp/crash-anaysis.OmalY/lustre.bin kmem: xarray_cache: slab: ffffea0004dbeb00 duplicate freelist object: ffff880136faefe8