************************ crashinfo ************************* /exports/testreports/48091/testresults/sanity2-ldiskfs-DNE-centos7_x86_64-centos7_x86_64/oleg122-client-timeout-core (3.10.0-7.9-debug) +==========================+ | *** Crashinfo v1.3.7 *** | +==========================+ +++WARNING+++ PARTIAL DUMP with size(vmcore) < 25% size(RAM) KERNEL: /tmp/crash-anaysis.tiKL7/vmlinux [TAINTED] DUMPFILE: /exports/testreports/48091/testresults/sanity2-ldiskfs-DNE-centos7_x86_64-centos7_x86_64/oleg122-client-timeout-core [PARTIAL DUMP] CPUS: 4 DATE: Fri Dec 20 05:45:06 EST 2024 UPTIME: 04:27:06 LOAD AVERAGE: 0.81, 0.76, 0.86 TASKS: 171 NODENAME: oleg122-client.virtnet RELEASE: 3.10.0-7.9-debug VERSION: #1 SMP Sat Mar 26 23:28:42 EDT 2022 MACHINE: x86_64 (2399 Mhz) MEMORY: 4 GB PANIC: "" +--------------------------+ >------------------------| Per-cpu Stacks ('bt -a') |------------------------< +--------------------------+ -- CPU#0 -- PID=0 CPU=0 CMD=swapper/0 #-1 native_safe_halt+0xb, 449 bytes of data #0 default_idle+0x1e #1 default_enter_idle+0x45 #2 cpuidle_enter_state+0x40 #3 cpuidle_idle_call+0xd8 #4 arch_cpu_idle+0xe #5 cpu_startup_entry+0x14a #6 rest_init+0x8e #7 start_kernel+0x456 #8 x86_64_start_reservations+0x2a #9 x86_64_start_kernel+0x152 #10 start_cpu+0x5 -- CPU#1 -- PID=0 CPU=1 CMD=swapper/1 #-1 native_safe_halt+0xb, 449 bytes of data #0 default_idle+0x1e #1 default_enter_idle+0x45 #2 cpuidle_enter_state+0x40 #3 cpuidle_idle_call+0xd8 #4 arch_cpu_idle+0xe #5 cpu_startup_entry+0x14a #6 start_secondary+0x1eb #7 start_cpu+0x5 -- CPU#2 -- PID=0 CPU=2 CMD=swapper/2 #-1 native_safe_halt+0xb, 449 bytes of data #0 default_idle+0x1e #1 default_enter_idle+0x45 #2 cpuidle_enter_state+0x40 #3 cpuidle_idle_call+0xd8 #4 arch_cpu_idle+0xe #5 cpu_startup_entry+0x14a #6 start_secondary+0x1eb #7 start_cpu+0x5 -- CPU#3 -- PID=0 CPU=3 CMD=swapper/3 #-1 native_safe_halt+0xb, 449 bytes of data #0 default_idle+0x1e #1 default_enter_idle+0x45 #2 cpuidle_enter_state+0x40 #3 cpuidle_idle_call+0xd8 #4 arch_cpu_idle+0xe #5 cpu_startup_entry+0x14a #6 start_secondary+0x1eb #7 start_cpu+0x5 +--------------------------------+ >---------------------| How This Dump Has Been Created |---------------------< +--------------------------------+ Cannot identify the specific condition that triggered vmcore +---------------+ >------------------------------| Tasks Summary |------------------------------< +---------------+ Number of Threads That Ran Recently ----------------------------------- last second 19 last 5s 30 last 60s 71 ----- Total Numbers of Threads per State ------ TASK_INTERRUPTIBLE 167 TASK_RUNNING 1 +++WARNING+++ There are 3 threads running in their own namespaces Use 'taskinfo --ns' to get more details +-----------------------+ >--------------------------| 5 Most Recent Threads |--------------------------< +-----------------------+ PID CMD Age ARGS ----- -------------- ------ ---------------------------- 11 rcuos/0 0 ms (no user stack) 9 rcu_sched 0 ms (no user stack) 17073 socknal_sd01_00 0 ms (no user stack) 30277 rm 0 ms rm -rf /mnt/lustre/d154f.sanity /mnt/lustre/d154g.sanity /mnt/lustre/d154i.sanity /mnt/lustre/d160a.sanity /mnt/lustre/d160c.sanity /mnt/lustre/d160d.sanity /mnt/lustre/d160f.sanity /mnt/lustre/d160g.sanity /mnt/lustre/d160h.sanity /mnt/lustre/d160i.sanity /mnt/lustre/d160j.sanity /mnt/lustre/d160k.sanity /mnt/lustre/d160l.sanity /mnt/lustre/d160m.sanity /mnt/lustre/d160n.sanity /mnt/lustre/d160o.sanity /mnt/lustre/d160s.sanity /mnt/lustre/d160t.sanity /mnt/lustre/d161a.sanity /mnt/lustre/d161b.sanity /mnt/lustre/d161c.sanity /mnt/lustre/d161d.sanity /mnt/lustre/d162a.sanity /mnt/lustre/d162b.sanity /mnt/lustre/d162c.sanity.local /mnt/lustre/d162c.sanity.remote /mnt/lustre/d165c.sanity /mnt/lustre/d165d.sanity /mnt/lustre/d165e.sanity-0 /mnt/lustre/d165e.sanity-1 /mnt/lustre/d182b.sanity /mnt/lustre/d183.sanity /mnt/lustre/d184a.sanity /mnt/lustre/d184b.sanity /mnt/lustre/d184c.sanity /mnt/lustre/d184d.sanity /mnt/lustre/d184e.sanity /mnt/lustre/d184f.sanity /mnt/lustre/d185.sanity /mnt/lustre/d187a.sanity /mnt/lustre/d187b.sanity /mnt/lustre/d2 /mnt/lustre/d200.pools /mnt/lustre/d204a.sanity /mnt/lustre/d204b.sanity /mnt/lustre/d204c.sanity /mnt/lustre/d204d.sanity /mnt/lustre/d204e.sanity /mnt/lustre/d204f.sanity /mnt/lustre/d204g.sanity /mnt/lustre/d204h.sanity /mnt/lustre/d205a.sanity.expire /mnt/lustre/d205a.sanity.remote /mnt/lustre/d205a.sanity.rename /mnt/lustre/d205b.sanity /mnt/lustre/d205h.sanity /mnt/lustre/d206.sanity /mnt/lustre/d220.sanity /mnt/lustre/d226a.sanity /mnt/lustre/d226b.sanity /mnt/lustre/d226c.sanity /mnt/lustre/d226d.sanity /mnt/lustre/d226d.sanity_03 /mnt/lustre/d228a.sanity /mnt/lustre/d228b.sanity /mnt/lustre/d228c.sanity /mnt/lustre/d230g.sanity /mnt/lustre/d230h.sanity /mnt/lustre/d230i.sanity /mnt/lustre/d230j.sanity /mnt/lustre/d230l.sanity /mnt/lustre/d230m.sanity /mnt/lustre/d230n.sanity /mnt/lustre/d230o.sanity /mnt/lustre/d230p.sanity /mnt/lustre/d230q.sanity /mnt/lustre/d230t.sanity /mnt/lustre/d230t.sanity.2 /mnt/lustre/d230w.sanity /mnt/lustre/d230x.sanity /mnt/lustre/d230z.sanity /mnt/lustre/d231a.sanity /mnt/lustre/d232a.sanity /mnt/lustre/d232b.sanity /mnt/lustre/d239A.sanity /mnt/lustre/d240.sanity /mnt/lustre/d242.sanity /mnt/lustre/d243.sanity /mnt/lustre/d244b.sanity /mnt/lustre/d247a.sanity /mnt/lustre/d247c.sanity /mnt/lustre/d247d.sanity /mnt/lustre/d247e.sanity /mnt/lustre/d247f.sanity /mnt/lustre/d247g.sanity /mnt/lustre/d247h.sanity /mnt/lustre/d248c.sanity /mnt/lustre/d254.sanity /mnt/lustre/d257.sanity /mnt/lustre/d270a.sanity /mnt/lustre/d270b.sanity /mnt/lustre/d270c.sanity /mnt/lustre/d270d.sanity /mnt/lustre/d270e.sanity /mnt/lustre/d270f.sanity /mnt/lustre/d270g.sanity /mnt/lustre/d270h.sanity /mnt/lustre/d270i.sanity /mnt/lustre/d270j.sanity /mnt/lustre/d271a.sanity /mnt/lustre/d271ba.sanity /mnt/lustre/d271b.sanity /mnt/lustre/d271c.sanity /mnt/lustre/d271d.sanity /mnt/lustre/d271f.sanity /mnt/lustre/d272a.sanity /mnt/lustre/d272d.sanity /mnt/lustre/d272e.sanity /mnt/lustre/d272f.sanity /mnt/lustre/d273a.sanity /mnt/lustre/d273b.sanity /mnt/lustre/d273c.sanity /mnt/lustre/d300a.sanity /mnt/lustre/d300b.sanity /mnt/lustre/d300g.sanity /mnt/lustre/d300h.sanity /mnt/lustre/d300i.sanity /mnt/lustre/d300k.sanity /mnt/lustre/d300l.sanity /mnt/lustre/d300n.sanity /mnt/lustre/d300p.sanity /mnt/lustre/d300r.sanity /mnt/lustre/d300s.sanity /mnt/lustre/d300t.sanity /mnt/lustre/d300ua.sanity /mnt/lustre/d300ub.sanity /mnt/lustre/d300ub.sanity.4 /mnt/lustre/d300ub.sanity.5 /mnt/lustre/d300uc.sanity /mnt/lustre/d300ud.sanity /mnt/lustre/d300ue.sanity /mnt/lustre/d300ug.sanity /mnt/lustre/d300uh.sanity /mnt/lustre/d300uj.sanity /mnt/lustre/d310a.sanity /mnt/lustre/d310b.sanity /mnt/lustre/d311.sanity /mnt/lustre/d316.sanity /mnt/lustre/d319.sanity /mnt/lustre/d360.sanity /mnt/lustre/d402.sanity /mnt/lustre/d406.sanity /mnt/lustre/d409.sanity /mnt/lustre/d412.sanity /mnt/lustre/d413a.sanity-fillmdt /mnt/lustre/d413a.sanity-s1 /mnt/lustre/d413b.sanity-s1 /mnt/lustre/d413c.sanity-s1 /mnt/lustre/d413d.sanity /mnt/lustre/d413e.sanity /mnt/lustre/d413f.sanity /mnt/lustre/d413g.sanity /mnt/lustre/d413h.sanity /mnt/lustre/d413i.sanity /mnt/lustre/d413j.sanity /mnt/lustre/d413k.sanity /mnt/lustre/d415.sanity /mnt/lustre/d417.sanity.1 /mnt/lustre/d418.sanity /mnt/lustre/d420.sanity /mnt/lustre/d421a.sanity /mnt/lustre/d421b.sanity /mnt/lustre/d421c.sanity /mnt/lustre/d421f.sanity /mnt/lustre/d421h.sanity /mnt/lustre/d422.sanity /mnt/lustre/d427.sanity /mnt/lustre/d428.sanity /mnt/lustre/d430a.sanity /mnt/lustre/d430b.sanity /mnt/lustre/d430c.sanity /mnt/lustre/d431.sanity /mnt/lustre/d432.sanity /mnt/lustre/d434.sanity /mnt/lustre/d442.sanity /mnt/lustre/d801b.sanity /mnt/lustre/d802b.sanity /mnt/lustre/d802c.sanity /mnt/lustre/d803a.sanity /mnt/lustre/d803b.sanity.0 /mnt/lustre/d803b.sanity.1 /mnt/lustre/d804.sanity /mnt/lustre/d820.sanity /mnt/lustre/d831.sanity /mnt/lustre/d832.sanity /mnt/lustre/d850.sanity /mnt/lustre/d851.sanity /mnt/lustre/d852.sanity /mnt/lustre/d904.sanity /mnt/lustre/f102ha.sanity /mnt/lustre/f102l.sanity /mnt/lustre/f102m.sanity /mnt/lustre/f102p.sanity /mnt/lustre/f102s.sanity /mnt/lustre/f102t.sanity /mnt/lustre/f105e.sanity /mnt/lustre/f121.sanity /mnt/lustre/f127a.sanity /mnt/lustre/f127b.sanity /mnt/lustre/f127c.sanity /mnt/lustre/f130h.sanity /mnt/lustre/f131c.sanity /mnt/lustre/f152.sanity /mnt/lustre/f154A.sanity /mnt/lustre/f154c.sanity.1 /mnt/lustre/f154c.sanity.2 /mnt/lustre/f154c.sanity.3 /mnt/lustre/f154d.sanity /mnt/lustre/f165b.sanity /mnt/lustre/f171.sanity /mnt/lustre/f205a.sanity /mnt/lustre/f205b.sanity.1 /mnt/lustre/f205c.sanity.1 /mnt/lustre/f205g.sanity.0001 /mnt/lustre/f209.sanity /mnt/lustre/f210.sanity /mnt/lustre/f213.sanity /mnt/lustre/f219.sanity /mnt/lustre/f219.sanity-2 /mnt/lustre/f224a.sanity /mnt/lustre/f224b.sanity /mnt/lustre/f224d.sanity /mnt/lustre/f235.sanity /mnt/lustre/f239a.sanity /mnt/lustre/f241a.sanity /mnt/lustre/f241b.sanity /mnt/lustre/f249.sanity /mnt/lustre/f250.sanity /mnt/lustre/f251b.sanity /mnt/lustre/f260.sanity /mnt/lustre/f277.sanity /mnt/lustre/f398e.sanity /mnt/lustre/f398e.sanity.new /mnt/lustre/f398k.sanity /mnt/lustre/f398l.sanity.2 /mnt/lustre/f398o.sanity /mnt/lustre/f398r.sanity /mnt/lustre/f398s.sanity /mnt/lustre/f410.sanity /mnt/lustre/f411a.sanity /mnt/lustre/f429.sanity /mnt/lustre/f60b.sanity /mnt/lustre/f61 /mnt/lustre/f61b.sanity /mnt/lustre/f63b.sanity /mnt/lustre/f64f.sanity /mnt/lustre/f66 /mnt/lustre/f72b.sanity-dg /mnt/lustre/f72b.sanity-du /mnt/lustre/f72b.sanity-fg /mnt/lustre/f72b.sanity-fu /mnt/lustre/f77l.sanity /mnt/lustre/f806.sanity /mnt/lustre/f807a.sanity /mnt/lustre/f807b.sanity /mnt/lustre/f808.sanity /mnt/lustre/f809.sanity /mnt/lustre/f810.sanity /mnt/lustre/f812a.sanity /mnt/lustre/f812b.sanity /mnt/lustre/f812c.sanity /mnt/lustre/f814.sanity /mnt/lustre/f814.sanity.cp /mnt/lustre/f815.sanity /mnt/lustre/f816.sanity /mnt/lustre/f819a.sanity /mnt/lustre/f819b.sanity /mnt/lustre/f81a.sanity /mnt/lustre/f81b.sanity /mnt/lustre/f903.sanity /mnt/lustre/f905.sanity 598 kworker/0:2 0 ms (no user stack) +------------------------+ >-------------------------| Memory Usage (kmem -i) |-------------------------< +------------------------+ PAGES TOTAL PERCENTAGE TOTAL MEM 955079 3.6 GB ---- FREE 862081 3.3 GB 90% of TOTAL MEM USED 92998 363.3 MB 9% of TOTAL MEM SHARED 10868 42.5 MB 1% of TOTAL MEM BUFFERS 2999 11.7 MB 0% of TOTAL MEM CACHED 37072 144.8 MB 3% of TOTAL MEM SLAB 12270 47.9 MB 1% of TOTAL MEM TOTAL HUGE 0 0 ---- HUGE FREE 0 0 0% of TOTAL HUGE TOTAL SWAP 262143 1024 MB ---- SWAP USED 256 1 MB 0% of TOTAL SWAP SWAP FREE 261887 1023 MB 99% of TOTAL SWAP COMMIT LIMIT 739682 2.8 GB ---- COMMITTED 76419 298.5 MB 10% of TOTAL LIMIT +-------------------------------+ >----------------------| Scheduler Runqueues (per CPU) |----------------------< +-------------------------------+ ---+ CPU=0 ---- | CURRENT TASK , CMD=swapper/0 ---+ CPU=1 ---- | CURRENT TASK , CMD=swapper/1 ---+ CPU=2 ---- | CURRENT TASK , CMD=swapper/2 ---+ CPU=3 ---- | CURRENT TASK , CMD=swapper/3 +------------------------+ >-------------------------| Network Status Summary |-------------------------< +------------------------+ TCP Connection Info ------------------- ESTABLISHED 6 TIME_WAIT 92 LISTEN 13 NAGLE disabled (TCP_NODELAY): 5 user_data set (NFS etc.): 8 UDP Connection Info ------------------- 15 UDP sockets, 0 in ESTABLISHED user_data set (NFS etc.): 4 Unix Connection Info ------------------------ ESTABLISHED 29 CLOSE 20 LISTEN 8 Raw sockets info -------------------- CLOSE 1 Interfaces Info --------------- How long ago (in seconds) interfaces transmitted/received? Name RX TX ---- ---------- --------- lo n/a 16023.2 eth0 n/a 0.0 RSS_TOTAL=104228 pages, %mem= 1.6 +------------+ >-------------------------------| Mounted FS |-------------------------------< +------------+ MOUNT SUPERBLK TYPE DEVNAME DIRNAME ffff880138cca000 ffff880139940800 rootfs rootfs / ffff880138ccb500 ffff88012aa0d000 sysfs sysfs /sys ffff880138ccb6c0 ffff880139944000 proc proc /proc ffff880138ccb880 ffff880137678000 devtmpfs devtmpfs /dev ffff880138ccba40 ffff8800b6ca1800 securityfs securityfs /sys/kernel/security ffff880138ccbc00 ffff88012aa0d800 tmpfs tmpfs /dev/shm ffff880138ccbdc0 ffff88013771f000 devpts devpts /dev/pts ffff88012ab2c000 ffff88012aa0e000 tmpfs tmpfs /run ffff88012ab2c1c0 ffff88012aa0e800 tmpfs tmpfs /sys/fs/cgroup ffff88012ab2c380 ffff88012aa0f000 cgroup cgroup /sys/fs/cgroup/systemd ffff88012ab2c540 ffff88012aa0f800 pstore pstore /sys/fs/pstore ffff88012ab2c700 ffff88012ab39800 cgroup cgroup /sys/fs/cgroup/freezer ffff88012ab2c8c0 ffff88012ab39000 cgroup cgroup /sys/fs/cgroup/hugetlb ffff88012ab2ca80 ffff88012ab38800 cgroup cgroup /sys/fs/cgroup/pids ffff88012ab2cc40 ffff88012ab38000 cgroup cgroup /sys/fs/cgroup/devices ffff88012ab2ce00 ffff88012ab3a000 cgroup cgroup /sys/fs/cgroup/cpu,cpuacct ffff88012ab2cfc0 ffff88012ab3a800 cgroup cgroup /sys/fs/cgroup/net_cls,net_prio ffff88012ab2d180 ffff88012ab3b000 cgroup cgroup /sys/fs/cgroup/blkio ffff88012ab2d340 ffff88012ab3b800 cgroup cgroup /sys/fs/cgroup/perf_event ffff88012ab2d500 ffff88012ab3c000 cgroup cgroup /sys/fs/cgroup/memory ffff88012ab2d6c0 ffff88012ab3c800 cgroup cgroup /sys/fs/cgroup/cpuset ffff88012ab2da40 ffff88012ab3e000 configfs configfs /sys/kernel/config ffff880137669500 ffff8800b6ca7000 ext4 /dev/nbd0 / ffff8800b6e5e380 ffff880137326000 rpc_pipefs rpc_pipefs /var/lib/nfs/rpc_pipefs ffff88013700e700 ffff88012a58f000 autofs systemd-1 /proc/sys/fs/binfmt_misc ffff88013700e8c0 ffff88012a58b800 hugetlbfs hugetlbfs /dev/hugepages ffff88013700ea80 ffff880139947800 debugfs debugfs /sys/kernel/debug ffff880137669c00 ffff88012b278800 mqueue mqueue /dev/mqueue ffff8800b6e5e000 ffff8800b638b000 binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc/ ffff88012ab2dc00 ffff8800b6fc6000 ramfs none /mnt ffff88012ab2ddc0 ffff88012a623000 tmpfs none /var/lib/stateless/writable ffff8800b61a2000 ffff8800b623e000 squashfs /dev/vda /home/green/git/lustre-release ffff880137669880 ffff88012a623000 tmpfs none /var/cache/man ffff88013700ee00 ffff88012a623000 tmpfs none /var/log ffff8800b61a21c0 ffff88012a623000 tmpfs none /var/lib/dbus ffff8800b61a2380 ffff88012a623000 tmpfs none /tmp ffff8801376696c0 ffff88012a623000 tmpfs none /var/lib/dhclient ffff8800b61a2540 ffff88012a623000 tmpfs none /var/tmp ffff8800b38f8000 ffff88012a623000 tmpfs none /var/lib/NetworkManager ffff8800b61a2700 ffff88012a623000 tmpfs none /var/lib/systemd/random-seed ffff8800b38f81c0 ffff88012a623000 tmpfs none /var/spool ffff8800b61a28c0 ffff88012a623000 tmpfs none /var/lib/nfs ffff8800b38f8380 ffff88012a623000 tmpfs none /var/lib/gssproxy ffff8800b38f8540 ffff88012a623000 tmpfs none /var/lib/logrotate ffff8800b38f8700 ffff88012a623000 tmpfs none /etc ffff8800b38f88c0 ffff88012a623000 tmpfs none /var/lib/rsyslog ffff8800b38f8a80 ffff88012a623000 tmpfs none /var/lib/dhclient/var/lib/dhclient ffff8800b6e5e540 ffff88012a53c800 nfs4 192.168.200.253:/exports/state/oleg122-client.virtnet /var/lib/stateless/state ffff8800b6e5e8c0 ffff88012a53c800 nfs4 192.168.200.253:/exports/state/oleg122-client.virtnet /boot ffff8800b6e5ea80 ffff88012a53c800 nfs4 192.168.200.253:/exports/state/oleg122-client.virtnet /etc/etc/kdump.conf ffff8800b6e5ec40 ffff880137326000 rpc_pipefs sunrpc /var/lib/nfs/var/lib/nfs/rpc_pipefs ffff88012b508700 ffff88012c0fc800 nfs4 192.168.200.253://exports/testreports/48091/testresults/sanity2-ldiskfs-DNE-centos7_x86_64-centos7_x86_64 /tmp/tmp/testlogs ffff88012b508fc0 ffff88012c0fd000 tmpfs tmpfs /run/user/0 ffff88013700fa40 ffff8800aaa57000 nfsd nfsd /proc/fs/nfsd ffff88013700fdc0 ffff8800b623e000 squashfs /dev/vda /usr/sbin/mount.lustre ffff88012f3c3500 ffff8801353f9800 lustre 192.168.201.122@tcp:/lustre /mnt/lustre +-------------------------------+ >----------------------| Last 40 lines of dmesg buffer |----------------------< +-------------------------------+ [15800.200365] LNet: Removed LNI 192.168.201.22@tcp [15803.756786] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [15803.763233] alg: No test for adler32 (adler32-zlib) [15804.704856] Lustre: Lustre: Build Version: 2.16.50_171_gcc1f609 [15804.928988] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [15804.931051] LNet: Accept secure, port 988 [15806.499747] Key type lgssc registered [15806.921149] Lustre: Echo OBD driver; http://www.lustre.org/ [15837.352276] Lustre: Mounted lustre-client [15839.551408] Lustre: DEBUG MARKER: Using TIMEOUT=20 [15847.600749] Lustre: DEBUG MARKER: == sanity test 901: don't leak a mgc lock on client umount ========================================================== 05:42:06 (1734691326) [15848.865658] Lustre: Unmounted lustre-client [15848.978671] Lustre: Mounted lustre-client [15850.883877] Lustre: DEBUG MARKER: == sanity test 902: test short write doesn't hang lustre ========================================================== 05:42:10 (1734691330) [15850.987903] Lustre: *** cfs_fail_loc=2001415, val=0*** [15852.949561] Lustre: DEBUG MARKER: == sanity test 903: Test long page discard does not cause evictions ========================================================== 05:42:12 (1734691332) [15858.511522] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms [15878.514737] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake [15878.528837] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms [15898.532712] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake [15898.545866] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms [15918.548664] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake [15918.567137] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms [15938.571674] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake [15938.591110] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms [15958.595660] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake [15958.609028] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms [15978.611735] LustreError: 20145:0:(osc_cache.c:3083:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake [15988.321066] Lustre: DEBUG MARKER: == sanity test 904: virtual project ID xattr ============= 05:44:27 (1734691467) [15991.709005] Lustre: DEBUG MARKER: == sanity test 905: bad or new opcode should not stuck client ========================================================== 05:44:31 (1734691471) [15992.033613] LustreError: lustre-OST0000-osc-ffff8801353f9800: operation ost_ladvise to node 192.168.201.122@tcp failed: rc = -95 [15993.629302] Lustre: DEBUG MARKER: == sanity test 906: Simple test for io_uring I/O engine via fio ========================================================== 05:44:32 (1734691472) [15994.118717] Lustre: DEBUG MARKER: SKIP: sanity test_906 Client OS does not support io_uring I/O engine [15994.587719] Lustre: DEBUG MARKER: == sanity test 907: write rpc error during unlink ======== 05:44:33 (1734691473) [15996.012792] LustreError: lustre-OST0000-osc-ffff8801353f9800: operation ost_write to node 192.168.201.122@tcp failed: rc = -3 [15996.015458] LustreError: Skipped 3 previous similar messages [15997.630389] Lustre: DEBUG MARKER: == sanity test 908a: llog created with valid ctime ======= 05:44:36 (1734691476) [15999.431803] Lustre: DEBUG MARKER: == sanity test 908b: changelog stores valid mtime ======== 05:44:38 (1734691478) [16012.780640] Lustre: DEBUG MARKER: == sanity test complete, duration 15861 sec ============== 05:44:51 (1734691491) [16013.387659] Lustre: DEBUG MARKER: === sanity: start cleanup 05:44:52 (1734691492) === ****************************************************************************** ************************ A Summary Of Problems Found ************************* ****************************************************************************** -------------------- A list of all +++WARNING+++ messages -------------------- PARTIAL DUMP with size(vmcore) < 25% size(RAM) There are 3 threads running in their own namespaces Use 'taskinfo --ns' to get more details ------------------------------------------------------------------------------ ** Execution took 10.71s (real) 5.54s (CPU), Child processes: 5.13s