00010000:00010000:0.0F:1715208060.838838:0:2469:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200080:2.0F:1715208060.966373:0:15371:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=23fb0f0 00800000:00000080:2.0:1715208060.966402:0:15371:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b5ee4c00 uarg=0000000000000000 00000002:00000080:2.0:1715208060.966412:0:15371:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b5ee4c00 uarg=0000000000000000 00000100:00100000:2.0:1715208060.966467:0:15371:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ce00 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15371:1798523505065728:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715208060.966625:0:15371:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aa4bcd80 going to sleep for 11 seconds 00000100:00100000:2.0:1715208060.968390:0:15371:0:(client.c:2956:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8800a9618000: committing for last_committed 4294967472 gen 1 00000100:00100000:2.0:1715208060.968404:0:15371:0:(client.c:3005:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY) req@ffff8800a9678000 x1798523505064512/t4294967472(4294967472) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715208073 ref 1 fl Complete:RPQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000100:00100000:2.0:1715208060.968424:0:15371:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ce00 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15371:1798523505065728:192.168.201.123@tcp:60:lhsmtool_posix.0 00000002:00000080:2.0:1715208060.968452:0:15371:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b5ee4c00 uarg=0000000000000000 00000100:00100000:2.0:1715208060.968517:0:15371:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ce00 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15371:1798523505065792:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715208060.968616:0:15371:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aa4bcd80 going to sleep for 11 seconds 00000100:00100000:1.0F:1715208060.970061:0:15371:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ce00 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15371:1798523505065792:192.168.201.123@tcp:60:lhsmtool_posix.0 00000080:00200008:1.0:1715208060.970113:0:15371:0:(file.c:430:ll_file_release()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:1.0:1715208060.970151:0:15371:0:(mdc_request.c:927:mdc_close()) @@@ matched open req@ffff8800a9678000 x1798523505064512/t4294967472(4294967472) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715208073 ref 1 fl Complete:RPQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000020:00100000:1.0:1715208060.970175:0:15371:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1715208060.970181:0:15371:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a9678a80 x1798523505065920/t0(0) o35->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:23/10 lens 392/840 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00000100:00100000:1.0:1715208060.970197:0:15371:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9678a80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15371:1798523505065920:192.168.201.123@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1715208060.970252:0:15371:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b3911780 going to sleep for 11 seconds 00000100:00100000:1.0:1715208060.972057:0:15371:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9678a80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15371:1798523505065920:192.168.201.123@tcp:35:lhsmtool_posix.0 00000002:00100000:1.0:1715208060.972091:0:15371:0:(mdc_request.c:837:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff8800a9678000 x1798523505064512/t4294967472(4294967472) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715208073 ref 1 fl Complete:RQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200008:1.0:1715208060.972150:0:15371:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200008:1.0:1715208060.976313:0:15371:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00010000:00010000:3.0F:1715208061.840711:0:5362:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1715208062.842808:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1715208063.826176:0:16615:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 25a: Restore lost file (HS_LOST flag) from import ========================================================== 18:41:03 (1715208063) 00010000:00010000:0.0:1715208063.844734:0:2527:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1715208063.959095:0:16658:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715208063.959106:0:16658:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715208063.959131:0:16658:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715208063.959153:0:16658:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715208063.959177:0:16658:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715208063.959194:0:16658:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715208063.959200:0:16658:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd5570590 00000080:00010000:3.0:1715208063.959212:0:16658:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd5570590 from it ffff88012b4a7cc8 00010000:00010000:3.0:1715208063.959219:0:16658:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715208063.959233:0:16658:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00000080:00200008:3.0:1715208063.959905:0:16658:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f400c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715208063.959911:0:16658:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715208063.959926:0:16658:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715208063.959978:0:16658:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.959987:0:16658:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.959997:0:16658:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 16658 initiator: MDT0 00000100:00100000:3.0:1715208063.960016:0:16658:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a927e780 going to sleep for 0 seconds 00000100:00100000:3.0:1715208063.960034:0:16658:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a91e6300 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:04532c2b-e5e9-420d-abb0-4480aab1e378:16658:1798523505065984:192.168.201.123@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1715208063.960045:0:16658:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.960053:0:16658:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.960059:0:16658:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.960065:0:16658:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.960072:0:16658:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.960078:0:16658:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800a9608240 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715208063.960083:0:16658:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.960089:0:16658:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.960096:0:16658:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.960102:0:16658:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715208063.960108:0:16658:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 16658 initiator: MDT0 00010000:00010000:3.0:1715208063.960114:0:16658:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608240/0x7561918bd5570662 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 16658 initiator: MDT0 00000080:00020000:3.0:1715208063.960135:0:16658:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:3.0:1715208063.960159:0:16658:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715208063.960163:0:16658:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715208063.960172:0:16658:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715208063.960180:0:16658:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715208063.960189:0:16658:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715208063.960196:0:16658:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715208063.960199:0:16658:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd5570590 00000080:00010000:3.0:1715208063.960203:0:16658:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd5570590 from it ffff88012b4a7cc8 00010000:00010000:3.0:1715208063.960207:0:16658:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715208063.960214:0:16658:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715208063.960297:0:16658:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715208063.960307:0:16658:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715208063.960314:0:16658:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715208063.960322:0:16658:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715208063.960330:0:16658:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715208063.960332:0:16658:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd5570590 00000080:00010000:3.0:1715208063.960336:0:16658:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd5570590 from it ffff88012b4a7b80 00010000:00010000:3.0:1715208063.960344:0:16658:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715208063.960352:0:16658:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715208063.960361:0:16658:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 44 00000080:00200008:3.0:1715208063.960369:0:16658:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200000:1.0:1715208063.961255:0:16659:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715208063.961272:0:16659:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715208063.961284:0:16659:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715208063.961296:0:16659:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715208063.961306:0:16659:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715208063.961309:0:16659:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd5570590 00000080:00010000:1.0:1715208063.961320:0:16659:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd5570590 from it ffff88012dc6fb80 00010000:00010000:1.0:1715208063.961324:0:16659:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715208063.961332:0:16659:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715208063.961343:0:16659:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 44 00000080:00200008:1.0:1715208063.961349:0:16659:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200000:1.0:1715208063.961376:0:16659:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715208063.961382:0:16659:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715208063.961389:0:16659:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715208063.961397:0:16659:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715208063.961404:0:16659:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715208063.961406:0:16659:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd5570590 00000080:00010000:1.0:1715208063.961409:0:16659:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd5570590 from it ffff88012dc6fbb0 00010000:00010000:1.0:1715208063.961412:0:16659:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715208063.961419:0:16659:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa669d40/0x7561918bd5570590 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878927e4 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715208063.961426:0:16659:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 1 00000080:00200000:1.0:1715208063.961458:0:16659:0:(dcache.c:318:ll_revalidate_dentry()) VFS Op:name=.lustre, flags=17 00000080:00200000:1.0:1715208063.961463:0:16659:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x1:0x0](ffff880136f410c8), inode mode 41ed mask 1 00000080:00200000:1.0:1715208063.961477:0:16659:0:(namei.c:1183:ll_atomic_open()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff880136f410c8), file ffff880136ac5980, open_flags 8000, mode 0 opened 0 00000080:00200000:1.0:1715208063.961485:0:16659:0:(namei.c:838:ll_lookup_it()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff880136f410c8), intent=open 00000002:00010000:1.0:1715208063.961496:0:16659:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: fid,[0x0:0x0:0x0]) in obj [0x200000002:0x1:0x0], intent: open flags 0100001 00010000:00010000:1.0:1715208063.961534:0:16659:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 16659 initiator: MDT0 00010000:00010000:1.0:1715208063.961541:0:16659:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 16659 initiator: MDT0 00000020:00100000:1.0:1715208063.961552:0:16659:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1715208063.961594:0:16659:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a9678a80 x1798523505066112/t0(0) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/66536 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00010000:00010000:1.0:1715208063.961606:0:16659:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 16659 initiator: MDT0 00000100:00100000:1.0:1715208063.961623:0:16659:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9678a80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:16659:1798523505066112:192.168.201.123@tcp:101:lhsmtool_posix.0 00000100:00100000:1.0:1715208063.961793:0:16659:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b3911cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1715208063.964032:0:16659:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9678a80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:16659:1798523505066112:192.168.201.123@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1715208063.964052:0:16659:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 16659 initiator: MDT0 00010000:00010000:2.0:1715208063.964063:0:16659:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 16659 initiator: MDT0 00010000:00010000:2.0:1715208063.964070:0:16659:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 16659 initiator: MDT0 00010000:00010000:2.0:1715208063.964076:0:16659:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 16659 initiator: MDT0 00010000:00010000:2.0:1715208063.964083:0:16659:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 16659 initiator: MDT0 00010000:00010000:2.0:1715208063.964099:0:16659:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa668480 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:2.0:1715208063.964103:0:16659:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 16659 initiator: MDT0 00010000:00010000:2.0:1715208063.964108:0:16659:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 16659 initiator: MDT0 00010000:00010000:2.0:1715208063.964114:0:16659:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 5/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 16659 initiator: MDT0 00010000:00010000:2.0:1715208063.964120:0:16659:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1715208063.964124:0:16659:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 16659 initiator: MDT0 00010000:00010000:2.0:1715208063.964129:0:16659:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668480/0x7561918bd5570669 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 16659 initiator: MDT0 00000002:00100000:2.0:1715208063.964146:0:16659:0:(mdc_locks.c:838:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff8800a9678a80 x1798523505066112/t4294967474(4294967474) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715208079 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000002:00100000:2.0:1715208063.964160:0:16659:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a9678a80 x1798523505066112/t4294967474(4294967474) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715208079 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:2.0:1715208063.964185:0:16659:0:(llite_lib.c:2872:ll_update_inode()) inode=[0x200000002:0x2:0x0], updating i_size 4096 00000080:00200000:2.0:1715208063.964188:0:16659:0:(namei.c:157:ll_iget()) got inode: [0x200000002:0x2:0x0](ffff8800b02788c8): rc = 0 00000080:00010000:2.0:1715208063.964194:0:16659:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000002:0x2:0x0](ffff8800b02788c8) for lock 0x0 00000080:00200008:2.0:1715208063.964208:0:16659:0:(file.c:871:ll_file_open()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:2.0:1715208063.964222:0:16659:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a9678a80 x1798523505066112/t4294967474(4294967474) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715208079 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:2.0:1715208063.964284:0:16659:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x2:0x0](ffff8800b02788c8), inode mode 4040 mask 44 00000080:00200080:2.0:1715208063.964338:0:16659:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=1f870f0 00800000:00000080:2.0:1715208063.964354:0:16659:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a977e540 uarg=0000000000000000 00000002:00000080:2.0:1715208063.964364:0:16659:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a977e540 uarg=0000000000000000 00000100:00100000:2.0:1715208063.964391:0:16659:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:16659:1798523505066176:192.168.201.123@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1715208063.964466:0:16659:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aa4bc240 going to sleep for 11 seconds 00000100:00100000:2.0:1715208063.965859:0:16659:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:16659:1798523505066176:192.168.201.123@tcp:59:lhsmtool_posix.0 00000002:00000080:2.0:1715208063.965881:0:16659:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a977e540 uarg=0000000000000000 00000100:00100000:2.0:1715208063.965914:0:16659:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:16659:1798523505066240:192.168.201.123@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1715208063.965961:0:16659:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aa4bc240 going to sleep for 11 seconds 00000100:00100000:2.0:1715208063.967622:0:16659:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:16659:1798523505066240:192.168.201.123@tcp:59:lhsmtool_posix.0 00000080:00200000:1.0:1715208063.971012:0:16660:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715208063.971027:0:16660:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715208063.971059:0:16660:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971073:0:16660:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971087:0:16660:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 16660 initiator: MDT0 00000100:00100000:1.0:1715208063.971103:0:16660:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b3911a80 going to sleep for 0 seconds 00000100:00100000:1.0:1715208063.971114:0:16660:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9678000 pname:cluuid:pid:xid:nid:opc:job mkdir:04532c2b-e5e9-420d-abb0-4480aab1e378:16660:1798523505066304:192.168.201.123@tcp:101:mkdir.0 00010000:00010000:1.0:1715208063.971126:0:16660:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971147:0:16660:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971155:0:16660:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971165:0:16660:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971175:0:16660:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971183:0:16660:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa668d80 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715208063.971189:0:16660:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971197:0:16660:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971207:0:16660:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971215:0:16660:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715208063.971221:0:16660:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 16660 initiator: MDT0 00010000:00010000:1.0:1715208063.971229:0:16660:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570670 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 16660 initiator: MDT0 00000080:00020000:1.0:1715208063.971255:0:16660:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000100:00080000:2.0:1715208063.992712:0:302:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715208063.992719:0:302:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715208063.992730:0:302:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715208063.992734:0:302:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715208063.992737:0:302:0:(pinger.c:253:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715208063.992740:0:302:0:(pinger.c:271:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:2.0:1715208063.992771:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2002:1798523505066432:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715208063.992842:0:2002:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1715208063.992850:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2002:1798523505066368:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715208063.992868:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523505066496:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715208063.994509:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523505066496:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715208063.994543:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2002:1798523505066432:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715208063.994604:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2002:1798523505066368:192.168.201.123@tcp:400:kworker.0 00000080:00200000:1.0:1715208064.122488:0:16697:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715208064.122517:0:16697:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715208064.122632:0:16697:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122649:0:16697:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122665:0:16697:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 16697 initiator: MDT0 00000100:00100000:1.0:1715208064.122690:0:16697:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b3911840 going to sleep for 0 seconds 00000100:00100000:1.0:1715208064.122705:0:16697:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9678000 pname:cluuid:pid:xid:nid:opc:job mkdir:04532c2b-e5e9-420d-abb0-4480aab1e378:16697:1798523505066560:192.168.201.123@tcp:101:mkdir.0 00010000:00010000:1.0:1715208064.122720:0:16697:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122731:0:16697:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122741:0:16697:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122751:0:16697:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122760:0:16697:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122769:0:16697:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa668d80 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715208064.122776:0:16697:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122786:0:16697:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122796:0:16697:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122814:0:16697:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715208064.122823:0:16697:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 16697 initiator: MDT0 00010000:00010000:1.0:1715208064.122849:0:16697:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668d80/0x7561918bd5570677 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 16697 initiator: MDT0 00000080:00020000:1.0:1715208064.122884:0:16697:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000001:02000400:3.0:1715208064.620672:0:16831:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_25a: @@@@@@ FAIL: cannot create directory '/mnt/lustre/d25a.sanity-hsm' 00010000:00010000:1.0:1715208064.846738:0:2469:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 198 lines, 198 kept, 0 dropped, 0 bad.